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I.  INTRODUCTION 


A.  PURPOSE 

A  program  manager  (PM)  is  confronted  by  many  risks  and  uncertainties  beyond 
the  common  cost,  schedule,  and  performance  ones.  Some  of  these  risks  are  unique  to  the 
DoD  acquisitions  environment  while  others  are  known  to  any  project  or  program 
attempting  to  leverage  the  benefits  of  Open  Architecture  (OA).  This  thesis  discusses 
types  of  OA-based  acquisition  risks  and  uncertainties,  and  explores  various  tools  and 
techniques  used  by  PMs  in  previously  successful  acquisition  programs.  Finally,  this 
research  recommends  several  strategies,  based  on  the  primary  and  secondary  research,  to 
mitigate  different  risks  and  uncertainties. 

B.  BACKGROUND 

The  2004  National  Military  Strategy  of  the  United  States  stressed  the  importance 
of  seamlessly  integrated  networks  of  complex  weapons  systems,  intelligence  platforms, 
and  C2  mechanisms  to  facilitate  joint  operability  of  the  DoD’s  service  components.  The 
Undersecretary  of  Defense  for  Acquisitions  Technology  and  Logistics  mandated  that  a 
-miodular,  open  systems  approach  shall  be  employed”  in  all  acquisition  programs  in  2003. 

Service  Oriented  Architecture  (SOA),  with  its  principles  of  reusable,  loosely 
coupled  and  autonomous  services  enabled  traditional  -stove  piped”  DoD  systems  to 
interact  and  share  data.  Cost  savings  could  be  achieved  through  software  reuse  and 
scalability,  thus  allowing  a  system  change  in  size  or  volume  more  easily  to  meet 
increased  user  demand  without  quality  degradation  or  other  critical  issues.  Commercial 
SOA  methods  were  subsequently  adapted  to  military  use;  OA  was  adopted  to  address 
unique  requirements  of  DoD  systems  such  as  infonnation  assurance,  secure  data 
exchange,  and  stringent  performance  thresholds  critical  to  systems  in  which  human  lives 
depend  upon. 
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C.  RESEARCH  OBJECTIVES 

Research  conducted  for  this  thesis  encompasses  several  objectives.  First,  risks  to 
PMs  in  the  DAS  ecosystem  are  identified.  This  ecosystem  represents  various 
organizations  involved  with  acquisitions,  ranging  from  Congress  down  to  a  program’s 
Risk  Project  Team,  along  with  the  environment,  consisting  of  rules,  regulations,  laws, 
and  customs  dictating  organizational  behaviors.  The  second  objective  is  to  ascertain  if 
there  is  an  agreed  upon  definition  of  uncertainty  in  the  DAS.  The  third  objective  is  to 
discover  if  using  an  OA  strategy  assists  or  hinders  an  acquisition  program.  In  addition, 
does  an  OA  strategy  expose  a  program  to  unique  risks  and  uncertainties.  Finally,  this 
thesis  attempts  to  discover  if  OA  has  delivered  promised  benefits  to  the  DAS. 

D.  RESEARCH  QUESTIONS 

By  answering  the  following  questions,  this  thesis  endeavors  to  aide  DAS 
leadership  in  gaining  a  deeper  understanding  of  the  impact  of  risk  and  uncertainty. 
Moreover,  this  thesis  attempts  to  develop  the  best  strategies  in  mitigating  negative 
consequences. 

1 .  Do  PMs  encounter  different  risks  at  different  stages  of  their  career? 

2.  What  are  risks  in  the  current  DAS  acquisitions  process  beyond  cost,  schedule, 
and  performance? 

3.  What  is  the  definition  of  uncertainty  in  the  DAS? 

4.  Has  OA  added  to  the  complexity  of  systems  development? 

5.  Have  regulations,  policies,  and  procedures  reduced  or  increased  complexity 
and  instability  when  using  an  OA  approach? 

6.  What  are  sources  of  uncertainty  in  an  OA  environment? 

7.  Has  SOA  and  OA  produced  desired  results  and  promised  efficiencies? 

E.  METHODOLOGY 

A  series  of  interviews  were  conducted  with  a  diverse  group  with  experience  in  Air 


Force,  Army,  and  Navy  acquisition  programs.  Most  interviewees  had  work  experience  in 
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at  least  one  PM  position;  all  had  DoD  acquisition  experience.  Some  interview  subjects 
are  currently  filling  various  positions  at  PEO-IWS  and  PEO-LMW. 

During  this  primary  research,  an  ethno  methodological  approach  was  taken  when 
interviewing  subjects  with  experience  in  DAS.  Interviews  were  conducted  with  minimal 
directive  questioning  because  the  interviewer  wanted  to  engage  the  subjects  in  a  free 
flowing  discussion  without  possible  personal  biases  on  the  part  of  the  interviewer. 
Common  topics  bought  up  spontaneously  during  an  interview  were  viewed  as  more 
powerful  than  answers  to  a  scripted  list  of  questions  because  answers  would  not  be 
limited  by  any  intentional  or  unintentional  framework  imposed  by  the  interviewer.  In  a 
free  flowing  discussion,  the  chances  of  an  interviewee  bringing  up  topics  similar  to  topics 
bought  up  by  other  interviewees  are  miniscule  unless  the  topic  itself  is  of  importance  to 
DoD  acquisition  professionals. 

Most  interviews  were  conducted  without  interruption  in  the  interviewee’s  offices 
at  their  convenience.  Interviews  with  PEO-IWS  and  PEO-LMW  personnel  were  done 
primarily  over  the  telephone  at  a  prearranged  time  with  the  exception  of  one  PEO-IWS, 
which  was  conducted  during  lunch  hour  at  a  conference.  This  was  the  only  interview  that 
was  rushed  and  ended  prematurely  while  all  other  interview  sessions  averaging  an  hour. 
The  main  goal  of  the  first  round  of  interviews  was  defining  elements  of  risk  and 
uncertainty  in  DAS  and  the  impact  of  OA  and  SOA. 

A  second  round  of  interviews  was  conducted  with  additional  questions  asked  and 
clarification  of  various  data  points  after  analysis  of  the  first  data  set.  For  the  second 
round,  the  primary  goal  was  to  explore  commonalities  found  during  the  initial  interview 
process.  Additional  infonnation  was  collected  from  prior  research  theses,  academic 
papers,  Inspector  General  (IG)  reports,  and  academic  and  business  literature. 

F.  SCOPE 

This  research  addresses  various  risks  and  uncertainties  inherent  in  DAS  and 
strategies  used  by  PMs  to  overcome  the  effects.  It  includes  a  literature  review  addressing 
risk,  the  measurement  of  risk,  OA,  SOA,  and  the  success  or  failure  of  SOA  in  the 
commercial  sector. 
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After  addressing  risk  and  uncertainty  issues,  and  reviewing  past  acquisition 
programs  this  research  explores  how  the  DoD  acquisition  cycle  mitigates  risks  while 
aligning  day  to  day  efforts  of  acquisition  professionals  as  discovered.  Interviewees  were 
selected  based  on  the  criteria  of  prior  or  current  experience  working  in  a  DoD 
acquisitions  organization.  Interview  subjects  not  currently  working  in  acquisitions  all 
teach  acquisition  management  at  the  Naval  Postgraduate  School.  Initial  interview 
questions  were  selected  after  conducting  research  using  current  literature  on  risk,  risk 
management,  and  OA,  and  SO  A  in  the  DoD. 

This  research  also  addresses  whether  SOA  has  been  successful  in  the  business 
world  and  references  complementary  research  conducted  by  another  student  researcher. 
Finally,  this  explores  whether  OA  and  SOA  has  had  a  net  positive  or  negative  impact  on 
DAS. 


G.  THESIS  ORGANIZATION 

This  thesis  is  organized  into  five  chapters  in  an  effort  to  present  a  sequential  flow  of 
information.  Chapter  I  provides  an  overview  with  regard  to  purpose,  methodology,  and 
scope.  In  this  section,  research  questions  and  objectives  are  identified.  Chapter  II 
provides  a  background  for  understanding  Navy  OA,  SOA,  MOSA,  and  Open  Systems. 
Similarities,  differences,  and  emphasis  between  them  are  highlighted.  An  overview  of 
contemporary  DAS  and  the  risk  types  that  the  different  assessments,  plans  and  milestones 
attempt  to  mitigate  is  also  provided  in  this  section.  Chapter  III  takes  the  data  gathered 
during  the  primary  interview  phase  in  conjunction  with  secondary  research  of  case 
studies,  papers  and  other  literature,  identifies  risk  areas  providing  the  biggest  challenges 
to  acquisitions  professionals.  A  detennination  will  be  made  if  OA  and  SOA  have  helped 
mitigate  or  increased  risk  and  uncertainty.  Moreover,  this  section  discusses  whether  OA 
and  SOA  have  provided  cost  savings  and  increased  efficiencies  by  analyzing  primary  and 
secondary  data.  Chapter  V  presents  conclusions,  study  shortcomings,  and 
recommendations  for  future  research. 
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II.  LITERATURE  REVIEW 


A.  RISK  AND  UNCERTAINTY 

1.  Definition 

The  Risk  Management  Guide  for  DoD  Acquisition  defines  risk  as  -a-  measure  of 
future  uncertainties  in  achieving  program  perfonnance  goals  and  objectives  within 
defined  cost,  schedule,  and  performance  constraints”  (DoD,  2006).  This  closely 
resembles  the  standard  definition  of  risk  in  project  management  practice  as  -an  uncertain 
event  or  condition  that,  if  it  occurs,  has  a  positive  or  negative  effect  on  at  least  one 
project  objective”  (Project  Management  Institute,  2008). 

Risks  contain  three  components: 

•  A  future  root  cause,  which,  if  eliminated  or  corrected,  prevents  a  potential 
consequence  from  occurring, 

•  A  probability  assessed  at  the  present  time  of  that  future  root  cause 
occurring,  and 

•  The  consequence  of  that  future  occurrence  (DoD,  2006). 

With  these  three  components,  it  is  obvious  that  risk  can  only  occur  in  the  future. 
Once  an  event  or  cause  has  come  to  pass  it  is  no  longer  a  risk  that  can  be  mitigated  but  an 
issue  that  needs  to  be  managed  (DoD,  2006). 

For  the  purposes  of  this  thesis,  uncertainty  is  the  state  of  not  being  sure  if  an 
event  will  occur.  It  can  best  be  defined  in  its  relationship  to  risk  with  this  analogy:  a 
coin  is  being  tossed  and  it  is  uncertain  if  the  coin  will  come  up  heads  or  tails.  To  the 
observer,  there  is  no  risk  unless  the  observer  bets  a  dollar  on  the  result;  the  risk  then  is 
either  gaining  or  losing  a  dollar  (Mun,  2010).  In  the  DAS,  a  PM  may  face  a  similar 
situation  in  that  next  year’s  budget  allocations  may  be  uncertain  but  risk  may  or  may  not 
exist  depending  upon  the  amount  of  money  currently  available  in  the  program’s  accounts 
compared  to  the  next  year’s  resource  requirements. 

Uncertainties  are  categorized  in  three  types:  known,  unknown  and  unknowable. 
Known  uncertainties  are  future  events  certain  to  occur,  such  as  cars  at  an  intersection  will 
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stop  at  a  red  light.  We  are  fairly  certain  all  cars  will  stop  for  a  red  light  and  at  times,  risk 
our  lives  betting  on  this  known  uncertainty  by  crossing  the  street.  On  rare  occasions,  a 
car  will  run  a  red  light,  so  even  though  this  uncertainity  is  known,  there  is  always  a 
chance  of  being  wrong.  The  unknown  is  what  we  do  not  know  but  is  possible  to  simulate 
and  will  become  known  through  the  passage  of  time,  events  and  action.  The  unknowable 
contains  so  much  risk  and  uncertainty  that  the  passage  of  time,  events,  or  action  may  not 
reduce  the  levels  of  risk  and  uncertainty.  For  example,  an  unknowable  event  is  an 
earthquake.  Even  after  an  earthquake  hits,  we  are  not  sure  the  time  of  the  next 
earthquake  (Mun,  2010). 

2.  Risk  Management 

Every  project  and  program  in  the  DAS  encounters  a  wide  array  of  risk  and 
uncertainties.  A  typical  example  is  the  Virginia  class  submarine  program,  which  saw 
increased  costs,  cut  budgets,  and  changes  to  the  basic  shipbuilding  plan.  With  risks  and 
uncertainties  a  constant  in  the  DAS,  risk  management  plays  an  important  role  in  helping  a 
program  meet  cost,  schedule,  and  perfonnance  goals. 

Risk  management,  a  subcategory  of  Project  Management,  is  defined  as  the 
-application  of  knowledge,  skills,  tools  and  techniques  to  project  activities  to  meet 
project  requirements”  and  is  accomplished  through  the  application  and  integration  of  the 
-project  management  processes  of  initiating,  planning,  executing,  monitoring  and 
controlling  and  closing”  (Project  Management  Institute,  2008).  Besides  risk 
management,  a  PM’s  time  and  effort  must  address  other  activities  and  requirements  such 
as  Project  Integration  Management,  Cost  Management,  and  Human  Resource 
Management. 

The  goal  of  project  risk  management  is  to  increase  the  probability  and  impact  of 
positive  events,  and  decrease  the  probability  and  impact  of  negative  events  in  the 
project.”  This  is  done  through  a  series  of  six  processes  of  Project  Risk  Management, 
Plan  Risk  Management,  Identify  Risks,  Perform  Qualitative  Risk  Analysis,  Perform 
Quantitaive  Risk  Analyis,  Plan  Risk  Responses,  and  Montior  and  Control  Risks  (Project 
Management  Institute,  2008).  In  practice,  different  PMs  approach  each  step  differently, 
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as  and  some  may  not  give  equal  weight  to  quantitative  risk  analysis  as  opposed  to  a 
qualitative  approach.  These  six  processes  will  be  valid  for  most  programs,  especially  in 
the  DoD,  as  can  be  seen  in  Figure  1.  It  that  shows  the  DoD  Risk  Management  Process 
along  with  the  six  processes  of  Project  Risk  Management. 


Risk 

Identification 


DoD  Risk  Management  Process 


1.  Plan  Risk 
Management 

2.  Identify  Risks 


\ 


Risk  Analysis 


3.  Perform  Qualitative  Analysis 

4.  Perform  Quantitat  ive  Analysis 


\ 


Risk  Tracking 


6.  Monitor  and 
Control  Risks 


5.  Plan  Risk  Responses 


6.  Monitor  and 
Control  Risks 


Risk  Mitigation 
Plan 

Implementation 


Figure  1 .  DoD  Risk  Management  Processess  with  Six  Processes  of  Project  Risk 

Management  (From  DoD,  2006). 


Although  risk  management  is  most  concerned  with  unknown  and  unknowable 
uncertainties,  it  is  most  useful  in  helping  mitigate  unknown  uncertainties  because 
unknowable  factors  can  be  hedged  by  insurance  or  risk  acceptance  (Mun,  2010). 

3.  Risk  Analysis  in  the  DoD 

Risk  analysis  assesses  the  impact  of  potential  risks  on  the  cost,  schedule,  and 
performance  of  a  program.  The  Risk  Management  Guide  for  DoD  Acquisition  does  not 
prescribe  specific  methods  or  tools  and  only  provides  general  guidance  and  accepted 
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practices  for  DoD  acquisition  professionals  to  follow.  It  is  important  to  understand  the 
commonly  accepted  tools  and  metrics  used  by  contemporary  DoD  acquisition 
professionals  for  this  thesis. 

Any  events  potentially  impacting  a  program  are  identified  and  assessed  in  relation 
the  likelihood  and  consequence  of  occurrence  are  shown  in  a  Risk  Reporting  Matrix. 
This  matrix  is  shown  in  Figure  2  and  reflects  both  the  likelihood  and  consequences  of  an 
event  occurring — grades  are  given  from  one  to  five,  with  one  being  the  least  likely  or 
consequential  and  five  being  most  probable  and  severe. 
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Figure  2.  Risk  Reporting  Matrix  (From  DoD,  2006). 

Comparing  the  two  grades  results  in  one  of  twenty-five  boxes  in  the  matrix. 
Results  in  the  green  shaded  area  contains  a  low  level  of  risk  (consequence)  and 
uncertainty  (likelihood).  The  yellow  shaded  areas  are  of  medium  and  significant  levels  of 
risk  while  uncertainty  is  found  in  the  red  shaded  area. 

The  Probability  of  Occurrence  is  the  sole  factor  used  in  detennining  the  likelihood 
of  a  risk  and  the  likelihood  of  occurrence  is  left  to  the  user’s  discretion. 
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Figure  3.  Levels  of  Likelihood  Criteria  (From  DoD,  2006). 


Levels  of  risk  are  directly  linked  with  cost,  schedule,  and  performance  risks  using 
the  metrics  of  technical  performance  and  supportability,  ability  to  meet  key  dates  and  key 
program  milestones,  and  the  percentage  of  any  budget  or  unit  production  cost  increase. 

B.  OPEN  SYSTEMS  IN  DEPARTMENT  OF  DEFENSE  ACQUISITIONS 

1.  DoD  Instructions  and  Guidance 

An  open  systems  approach  was  introduced  into  DAS  with  the  publication  of  the  2003 
DoDD  5000. 1.  The  latest  version  of  the  directive  leaves  the  original  instruction 
unchanged: 

Acquisition  programs  shah  be  managed  through  the  application  of  a 
system  engineering  approach  that  optimizes  total  systems  performance  and 
minimizes  total  ownership  costs.  A  modular,  open  systems  approach  shah 
be  employed,  where  feasible.  (Undersecretary  of  Defense  [AT&L],  2007) 

In  2004,  the  Undersecretary  of  Defense  for  Acquisitions  Technology  and  Logistics 
(AT&L)  designated  the  Open  Systems  Joint  Task  Force  (OSJTF)  as  his  lead  for  a 
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Modular  Open  System  Approach  (MOSA)  in  the  DoD  (Undersecretary  of  Defense 
[AT&L].,  2004).  DoDD  5000.1  was  ambiguous  when  it  stated  that  a  MOSA  -shall  be 
employed,  where  feasible,”  though  DoDI  5000.2,  Operation  of  the  Defense  Acquisition 
System  made  MOSA  mandatory: 

Program  managers  shall  employ  MOSA  to  design  for  affordable  change, 
enable  evolutionary  acquisition,  and  rapidly  field  affordable  systems  that 
are  interoperable  in  the  joint  battle  space. 

OSJTF’s  mission  was  to  make  MOSA  an  integral  part  of  the  acquistion  process,  provide 
expert  assistance  in  applying  MOSA,  ensure  application  of  MOSA  by  all  acquisition 
programs,  and  colloborate  with  industry  to  ensure  a  viable  open  standards  base.  The 
mission  statement  along  with  directives  and  definitions  of  MOSA  and  open  systems 
terms  is  found  on  their  website:  (http://www.acq.osd.mil/ositf/mission.html). 

OSJTF  initially  produced  a  Program  Manager’s  guide  (OSJTF,  2004);  however,  by  201 1 
OSJTF  ceased  operations.  The  Office  of  the  Undersecretary  of  Defense  for  Systems 
Engineering  assumed  all  OSJTF,  and  is  now  the  DoD  lead  for  MOSA.  OSJTF 
terminology  and  principles  for  all  aspects  of  open  systems  is  still  in  use  throughout  the 
DoD  and  form  the  basis  for  many  definitions  in  this  thesis. 

2.  Open  Architecture 
a.  Definition 

An  open  architecture  employs  open  standards  for  key  interfaces  within  a 

system  (Open  Systems  Joint  Task  Force  [OSJTF],  n.d.).  Within  the  DoD,  the  U.S.  Navy 

is  the  leading  proponent  of  OA,  and  in  late  2002,  established  a  new  Program  Executive 

Office  (PEO)  for  Integrated  Warfare  Systems  (PEO-IWS).  PEO-IWS  is  responsible  for 

both  surface  and  submarine  combat  systems,  most  missile  systems  and  launchers,  guns, 

and  electronic  warfare  sytems.  Additionally,  PEO-IWS  is  responsible  for  integrating  all 

antisubmarine  warfare  systems  across  all  PEOs.  PEO-IWS  is  now  assigned  overall 

responsibility  for  directing  the  Navy’s  OA  effort  and  established  an  OA  Enterprise  team 

(OAET)  comprising  lead  PEOs  from  the  Navy’s  various  warfare  areas  (Air,  C4I, 

Subsurface,  Surface,  and  Space).  Subsequently,  the  Office  of  the  Chief  of  Naval 

Operations  (OPNAV)  directed  the  implementation  of  OA  principles  across  the  Navy  with 
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the  goal  of  delivering  -timely,  affordable,  interoperable  warfighting  capability  to  the 
fleet,  made  sustainable  by  the  flexible  integration  of  emerging  capabilities”  (OPNAV 
[N6/N7],  2005). 

3.  Principles  of  Navy  OA 

The  following  are  the  Principles  of  Naval  OA  (OPNAV  [N6/N7],  2005): 

a.  Modular  Design  and  Design  Disclosure 

Pennits  evolutionary  design,  technology  insertion,  competitive  innovation, 
and  alternative  competitive  approaches  from  multiple  qualified  sources. 

Traditionally,  many  weapon  systems  have  been  procured  from  a  single 
vendor,  which  decreases  compatibility  issues  but  risks  greater  costs.  The  Navy  hopes  to 
benefit  from  significant  costs  savings  that  are  possible  in  a  competitive  environment. 

b.  Reusable  Application  Software 

Selection  through  open  competition  of  -best  of  breed”  candidates 
reviewed  by  subject  matter  expert  peers  and  based  on  data  driven  analyses  and 
experimentation  to  meet  operational  requirements.  Design  disclosure  must  be  made 
available  for  evolutionary  improvement  to  all  qualified  sources. 

As  software  development  accounts  for  an  ever  increasingly  greater 
percentage  of  total  ownership  costs  in  DAS,  the  ability  to  reuse  software  could  allow 
significant  savings  in  development  costs  for  other  programs. 

c.  Interoperable  Joint  Warfighting  Applications  and  Secure 
Information  Exchange 

Use  of  common  services,  common  warfighting  applications,  and 
information  assurance  as  intrinsic  design  elements. 

Ease  of  interoperability  between  different  services  and  allies  along  with 
the  secure,  reliable,  and  timely  exchange  of  vital  data  are  key  performance  parameters  for 
the  DoD  and  form  the  backbone  of  modem  tactics  and  strategy. 
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Requirements  for  secure  information  exchange  is  one  of  the  major  design 
challenges  with  OA.  The  methods  and  techniques  ensuring  interoperability  are  almost 
diametrically  opposed  to  ones  keeping  a  system  secure  as  interfaces  and  services  from 
other  systems  open  additional  vectors  for  a  network -based  attack.  Although  there  are 
many  commercial  systems  such  as  medical  and  baking  systems  requiring  high  levels  of 
security,  DoD  weapons  systems  must  be  designed  to  operate  in  a  battlefield  environment 
with  lives  at  stake. 

d.  Life-Cycle  Affordability 

In  system  design  development,  delivery  and  support  while  mitigating 
commercial  off  the  shelf  (COTS)  obsolescence  by  exploiting  the  Rapid  Capability 
Insertion  Process/Advanced  Processor  Build  methodology  (RCIP/APB). 

RCIP  is  a  continuous,  reduced  cost  upgrade  of  hardware  and  software  by 
bypassing  expensive  development  costs  and  using  COTS  technology  wherever  feasible. 
In  the  past,  DoD  systems  have  taken  relatively  longer  to  field  to  operational  forces, 
compared  with  new  technology  introduction  into  the  commercial  marketplace.  By  the 
time  some  systems  are  fielded,  they  are  already  technologically  obsolete  to  what  is 
available  in  the  civilian  space. 

First  developed  by  the  Acoustic  Rapid  COTS  Insertion  (A-RCI)  program, 
APB  is  a  disciplined  process  to  take  new  functionality  and  algorithms  from  the  laboratory 
to  the  Fleet  in  under  two  years  per  Fleet  requirements.  APB  methodology  requires 
annual  software  updates  to  respond  to  changing  requirements  and  advances  in  software 
capabilities  while  also  relying  on  periodic  technology  insertions  to  maintain  computing 
capacity  at  state  of  the  practice  values  and  mitigate  against  COTS  obsolesence  issues 
(Kerr,  2005). 


e.  Competition  and  Collaboration 

Alternative  solutions  and  sources  creates  competition  between  vendors, 
keeping  costs  lower  than  if  the  DoD  relied  on  a  single  source.  Collaboration  pertains  to 
both  business  and  the  DoD  working  together  or  to  multiple  contractors  leveraging  their 


12 


areas  of  expertise  to  solve  a  problem.  One  organization  may  be  able  to  produce  better 
software  at  a  lower  cost  and  a  more  efficient  timeframe  than  another  just  specializing  in 
hardware.  Through  collaboration,  both  organizations  benefit  from  cutting-edge 
technology  without  incurring  the  expenses  and  production  time  required  to  develop 
alternative  solutions. 

A  recent  change  in  acquisition  strategy  is  to  encourage  competition  and 
collaboration  through  the  introduction  of  small  and  mediums  sized  firms  (SME)  into  the 
acquisition  ecosystem.  Recently,  the  National  Institute  of  Standards  and  Technology 
created  the  Technology  Innovation  Program  to  -accelerate  innovation  through  high  risk, 
high  reward  research  in  areas  of  critical  need”  with  the  intent  to  provide  grants  to  small 
and  medium  sized  firms  (Schacht,  2011). 

4.  Essential  OA  Performance  Characteristics 

MUIRSS  describes  essential  OA  performance  characteristics  of 
Maintainability,  Upgradeability,  Interfaces/Interoperability,  Reliability,  Safety,  and 
Security.  These  characteristics  are  mainly  derived  from  the  basic  principles  of  Navy  OA 
(Naegle,  B.  ,  2006). 


a.  Maintainability 

Ability  to  maintain  the  system  over  the  typically  long  life  cycle  of  DoD 
systems  is  directly  associated  with  the  OA  principle  of  Life  Cycle  Affordability. 

b.  Upgradeability 

Supports  OA  principles  of  Reusability  and  Life  Cycle  Affordability. 
Upgradeability  is  a  key  facilitator  of  future  software  reuse. 

c.  Interfaces/Interoperability 

Directly  associated  with  the  Modular  Design  and  Design  Disclosure , 
Interoperable  Joint  Warfighting  Applications  principle  and  the  Collaboration  principles. 
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d.  Reliability 

Closely  associated  with  the  principle  of  Interoperable  Joint  Warfighting 
Applications,  although  the  emphasis  is  on  interoperability  as  opposed  to  reliability.  It  is 
well  understood  that  the  DAS’s  highest  priority  should  be  given  to  delivering  systems 
ensuring  success  of  the  war  fighter  on  the  battlefield.  Systems  need  to  be  designed  to 
operate  under  battlefield  conditions  with  the  highest  levels  of  reliability. 

e.  Safety 

Safety  is  not  directly  addressed  in  any  of  the  principles  of  OA  but  is 
always  a  concern  with  any  DoD  system. 

/.  Security 

Since  a  breach  in  system  security  can  result  in  significant  damage  to  the 
interests  of  the  United  States  and  /  or  significant  loss  of  life,  there  is  a  stronger  emphasis 
on  the  security  of  systems  in  the  DoD  compared  to  the  commercial  world  where  security 
breaches  are  usually  measured  only  in  monetary  costs.  This  characteristic  is  directly 
associated  with  the  Navy  OA  principle  of  Secure  Information  Exchange. 

5.  Service  Oriented  Architecture 

SOA  has  played  an  increasingly  important  role  in  technology  development  in  the 
business  world  and  its  principles  and  methods  have  found  its  way  into  the  DAS. 

a.  Definition 

There  is  not  an  agreed  upon  definition  for  SOA  in  the  commercial  sector. 
In  a  complementary  thesis,  the  researcher  found  that  definitions  for  SOA  varied  from 
company  to  company.  For  example,  Hewlett-Packard  defined  SOA  as  an  architectural 
approach  of  loosely  coupled,  reusable,  standards  based,  and  well  defined  software 
services  easily  discoverable  by  other  applications  or  end-users  on  a  network.  IBM’s 
definition  is  a  set  of  linked  services  or  repeatable  business  tasks  that  can  be  accessed 
when  needed  over  a  network.  The  Business  Transfonnation  Agency’s  defines  SOA  as  an 
environment  in  tenns  of  shared  mission  and  business  functions  and  the  services  enabling 
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them  (Wolff,  2011).  However,  all  definitions  empahsize  the  importance  of  services 
easily  located  by  other  services  over  a  network. 

b.  SOA  in  the  DoD 

The  military  does  not  have  its  own  definition  of  SOA  nor  is  it  mentioned 
in  the  many  instructions  mandating  an  open  systems  approach  in  DoD  acquisitions.  The 
lack  of  definition  or  mandate  has  not  stopped  the  military  from  using  SOA  where 
feasible.  PEO-IWS  has  been  a  proponant  of  SOA  solutions  given  that  the  service  and 
network  based  approach  of  SOA  fits  neatly  into  key  Navy  OA  principles  (i.e., 
modularity,  reusability  and  interoperability).  A  major  challenge  remains  of  designing  a 
network  seamlessly  integrating  existing,  planned  and  future  platforms  and  systems  into  a 
secure,  fully  interoperable,  near  real  time  information  system  able  to  accomondate 
complex  -legacy”  or  traditionally  -stove  piped”  systems  that  may,  or  may  not,  have  been 
designed  with  interoperability  in  mind  (Naegle,  2006).  Recently,  limited  test  experiment 
proved  SOA  benefits  when  operating  under  a  traditional  -stovepiped”  DoD  technology. 
The  test  created  a  simulated  tactical  environment  based  on  AEGIS  sytems,  exchanging 
data  in  a  C2  environment  centered  around  open  source  SOA.  The  test  was  successful 
with  the  creation  of  resuable  services,  avoidance  of  complex  software  and  larger  code, 
and  the  services  operating  no  matter  the  source,  type  of  data  or  the  protocols  used 
(Fisher,  2011). 


6.  Open  Systems 

a.  Definition 

Systems  employing  modular  design,  uses  widely  supported  and  consensus 
based  standards  for  its  key  interfaces,  and  has  been  subjected  to  successful  validation 
tests  to  ensure  the  openness  of  its  key  interfaces  (Open  Systems  Joint  Task  Force 
[OSJTF],  n.d.). 

Modular  design  uses  popular  standards  allowing  a  system  to  benefit  from 
a  high  dregree  of  portability,  a  key  metric  describing  the  ease  in  which  the  software  can 
be  reused  along  with  ease  of  connectivity  and  interoperability  with  other  programs. 
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Popular  standards  also  make  it  easier  to  design  a  scalable  program  ,  allowing  a  system  to 
adapt  to  future  increased  demands  and  user  requirements.  By  building  flexibility  into 
acquisition  development  products  with  modularity  and  standardized  key  interfaces  Open 
Systems  allow  flexibility  in  design  helping  to  mitigate  technological  uncertainty  (Dillard 
&  Ford,  2008). 


b.  Impact  of  Open  Systems  on  the  DoD  Acquisition  System 

A  recent  acquisition  research  paper  described  the  impact  of  adopting  and 
using  open  systems  in  the  DAS.  The  paper  identified  important  acquisition  activities, 
assessed  the  impact  of  programs  using  an  evolutionary  acquisition  process,  and 
concluded  that  there  were  risks.  These  risks  include  reduced  DoD  control  over  standards, 
increased  standards-selection  risk  due  to  ongoing  evolution  of  standards,  and  increased 
testing  requirements  due  to  integration  of  evolving  commercial  items  into  DoD  systems. 
Alternatively,  design  risks  are  reduced  as  components,  subsystems,  and  systems  are  made 
consistent  with  established  standards.  In  summary,  use  of  open  systems  to  evolutionary 
acquisition  programs  tended  to  trade  away  design  risk  for  increased  standards  integration 
risk  (Dillard  &  Ford  2008). 

One  of  the  biggest  reasons  leading  to  standards  integration  risk  is  that 
modern  software  engineering  is  far  from  mature,  as  compared  to  other  industries  such  as 
aviation  or  the  automotive  industry.  There  are  few  commonly  agreeded  upon  standards 
for  languages,  tools,  architectures,  resue,  or  procedures.  To  meet  DAS  complexities, 
there  has  been  a  general  move  away  from  the  ADA  programming  language;  a  language 
still  widely  used  in  many  legacy  DoD  systems  to  languages  commonly  used  in 
commercial  industry  such  as  C++.  Today’s  commonly  agreed  upon  standards  that  are 
core  to  SOA,  MOSA,  and  Open  Systems  may  become  obsolete  in  the  future. 

Practices  and  procedures  enabling  successful  launches  of  sofware  products 
in  the  commercial  world  may  be  ineffective  in  developing  long-lived  DoD  software 
intensive,  warfighting  systems.  DoD  systems  are  designed  to  have  a  very  long  life  span 
as  opposed  to  commercially  based  software  designs  (Naegle,  2006).  The  USN’s  premier 
AEGIS  combat  system,  for  example,  was  first  tested  in  1973  and  is  still  in  widespread 
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use  around  the  world.  The  AEGIS  system  is  ancient  compared  to  the  relatively  short 
thirteen-year  life  span  of  the  Netscape  Navigator  web  browser. 

7.  MOSA 

a.  Definition 

An  integrated  business  and  technical  strategy  employing  a  modular  design 
and,  where  appropriate,  defines  key  interfaces  using  widely  supported,  consensus-based 
standards  that  are  published  and  maintained  by  a  recognized  industry  standards 
organization.  (Open  Systems  Joint  Task  Force  [OSJTF],  n.d.) 

b.  MOSA  in  the  DoD 

DoD  5000.1  mandates  MOSA’s  as  both  a  technical  and  organizational 
strategy  for  systems  development  and  systems  upgrades.  The  emphasis  is  on 
evolutionary  acquisition  and  spiral  software  development  using  popular  open  standards 
with  the  goal  of  cheaper  systems  integration.  The  OSJTF  was  established  to  oversee  this 
policy  until  its  functions  and  responsibilities  were  transferred  to  the  Assistant  Secretary 
of  Defense  for  Systems  Engineering. 

The  DoD  Acquisition  Guidebook  advises  that  PMs  summarize  their 
overall  MOSA  implementation  plan  throughout  their  program’s  life  cycle. 

8.  Relationship  between  OA,  SO  A,  MOSA  and  Open  Systems 

Modularity  and  a  standards  based  methodology  are  the  two  key  characteristics 
shared  by  all  the  Open  System  architectures.  Reusability  and  common  services  or 
interfaces  are  also  common  characteristics  between  the  systems. 

Table  1  compares  the  principles  of  Navy  OA  with  key  words  from  the  definitions 
and  principles  of  the  other  three  architectures  OA  was  derived  from.  This  table  not  only 
reveals  subtle  differences  in  emphasis  of  key  principles  and  strategies  but  also  provides  a 
visual  reference  on  principles  that  were  developed  to  meet  the  needs  of  the  Navy  and 
DoD  for  their  unique  systems  and  acquisition  environment.  For  other  architectures,  the 
principles  of  affordability  and  collaboration  are  usually  not  mentioned  in  the  definitions 
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or  emphasized  in  the  literature  though  both  principles  are  unspoken  goals.  Security  is  not 
mentioned  in  other  architectures  and  reflects  the  DoD’s  concern  of  keeping  data  secure 
from  potential  adversaries.  For  MOSA,  SOA,  and  Open  Systems  the  focus  is  on 
development  of  a  system  as  a  whole  and  their  basic  principles  will  remain  constant  for 
any  code  meant  to  be  secure  (i.e.,  such  as  medical  or  financial  software)  to  code  that  is 
not  meant  to  be  secure.  A  strategy  seeking  competition  from  potential  vendors  to  reduce 
costs  is  unique  to  Navy  OA. 
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Principles  Navy 

OA 

Open  Systems 

SOA 

MOSA 

Modular 

Modular  design 

Modular 

Reusable 

Widely  supported, 

consensus  based 

standards  for  key 

interfaces. 

Reusable  (Hewlett 

Packard) 

Interoperable 

Widely  supported, 

consensus  based 

standards  for  key 

interfaces. 

Standards  based 

(Hewlett  Packard). 

Shared  functions 

(BTA). 

Linked  services 

(IBM). 

Consensus  based 

standards. 

Secure 

Affordable 

Modularity  leading 

to  cheaper  systems 

integration. 

Competition 

Collaboration 

Services  easily 

discoverable  on  a 

network. 

Table  1.  Comparison  of  OA  with  Opens  Systems,  SOA,  and  MOSA. 
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9.  Risk  and  Small-  to  Medium-Sized  Enterprises 

Large  defense  companies  have  complied  with  the  DoD’s  low-risk  requirements 
primarily  due  to  large  monetary  resources.  However,  these  same  requirements  eliminate 
much  of  the  potential  competition  and  innovation  available  in  small  to  medium  sized 
enterprises  (SMEs).  Former  NAVSEA  commander,  Paul  Sullivan  stated  that  the  strategy 
of  the  open  business  model  -will  bring  small  companies  that  are  low  cost  and  highly 
talented  to  the  table  to  go  and  do  a  lot  of  this  work  for  the  Navy”  (Fein,  2006).  To 
capitalize  on  these  potential  benefits,  the  Navy’s  acquisitions  program  will  need  to 
drastically  alter  its  definition  of  acceptable  risk,  because  the  -Tow  cost,  highly  talented 
companies”  cannot  provide  the  same  level  of  risk  accommodation  that  larger  contractors 
have  done  over  the  years.  Corporations  take  greater  risks  to  garner  potentially  larger 
rewards;  the  DoD  acquisition  community  has  no  real  incentive  to  take  risks  because  it  is 
not  profit  driven.  As  such,  current  DoD  acquisition  programs  conversely  incentivizes 
risk  suppression,  a  counterproductive  strategy  when  attempting  to  foster  an  environment 
encouraging  innovation. 

If  innovation  is  a  motivation  for  utilizing  an  open  architecture/business  model, 
then  the  DoD  faces  a  potentially  crippling  constraint  from  budget  reductions.  The  overall 
federal  R&D  spending  was  reduced  3.5  percent  from  2010,  but  90.3  percent  of  those  cuts 
came  from  the  DoD  (Hardy,  2011).  Moving  towards  an  open  business  model  allowing 
innovative  SMEs  into  the  acquisitions  process  is  one  way  of  combating  budget  cuts  since 
competition  drives  down  the  contract  price.  The  SME  business  environment  creates  the 
necessity  for  innovation  because  the  -raison  d’etre  of  SMEs  is  to  develop  radical 
innovations  that  could  make  them  more  competitive  in  a  market  that  is  dominated  by 
large  firms  and  attractive  for  takeover  by  large  firms”  (Oke,  Burke,  &  Myers,  2007). 

Perhaps  the  reason  why  SMEs  are  more  successful  innovators  is  due  to  its  use  of 
intellectual  capital.  Intellectual  capital  is  generally  defined  as  all  intangible  assets,  both 
potential  and  realized,  contributing  to  a  company’s  success  (Housel  &  Lorentz,  2011; 
Grajkowska,  2011).  The  technological  growth  and  importance  of  innovation  has 
spawned  a  realization  that  intellectual  capital  is  -a-  critical  force”  for  growth  in  our  new 
economy  (Kamukama,  Ahiauzu,  &  Ntayi,  2010).  Because  SMEs  rely  on  intellectual 
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capital  more,  it  makes  sense  that  these  types  of  companies  better  utilizing  intangible 
assets  are  poised  for  greater  success.  These  IC  assets  must  -he  effectively  managed  in 
order  to  offer  companies  a  source  of  competitive  advantage”  (Cohen  &  Kaimenakis, 
2007).  Innovative  SMEs  surviving  in  the  volatile,  high-risk  marketplace  seem  to  have 
proven  they  are  effective  managers  of  intellectual  capital. 

Uncertainty  in  investing  in  SMEs  is  created  by  the  lack  of  competent  methods  to 
value  intellectual  capital  and  predict  future  success  based  on  it.  This  creates  an  enormous 
obstacle  for  DAS  personnel  to  overcome  as  part  of  the  new  challenge  implicit  in  moving 
towards  an  open  business  model.  With  little  track  record  and  no  book  value  to  rely  on, 
the  DoD  will  have  to  develop  metrics  to  value  SMEs  based  on  their  primary  asset  of 
intellectual  capital.  Intellectual  capital  is  so  difficult  to  evaluate  in  a  company  because  it 
is  intangible;  it  is  hard  to  assess  the  mental  capacity  and  innovation  of  individual 
employees,  let  alone  the  entire  company.  Particularly  relevant  in  software  firms, 
individual  employees  represent  the  greatest  asset  and  the  greatest  risk.  Innovative  SMEs 
are  required  to  invest  most  of  their  limited  capital  in  these  employees,  as  they  are  the 
drivers  of  production  and  must  be  kept  satisfied  so  they  do  not  leave  and  take  their  IC  to 
another  corporation  (Housel  &  Lorentz,  2011;  Swart  &  Kinnie,  2003).  A  SMEs’  reliance 
on  human  capital  and  other  intangible  assets  eliminates  the  effectiveness  of  traditional 
accounting  techniques  have  in  detennining  their  current  value  and  future  success. 

C.  THE  DOD  ACQUISITIONS  SYSTEM 

DAS’s  mission  is  to  meet  the  goals  of  the  National  Security  Strategy  by  managing 
technologies,  programs,  and  product  support  needed  to  equip  the  U.S.  Armed  Forces.  It 
is  a  management  process  by  which  the  Department  of  Defense  provides  effective, 
affordable,  and  timely  systems  to  the  users  (Undersecretary  of  Defense,  [AT&L],  2007). 
A  knowledge -based  approach  is  used  to  review  each  program  at  key  points  in  the 
acquisition  process  to  reduce  risks  while  milestones  are  set  up  throughout  the  cycle  with 
specific  criteria  a  system  must  pass  through  before  being  allowed  to  proceed. 
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1.  DoD  Decision  Support  Systems 

The  DoD  Decision  Support  Systems  is  the  environment  within  which  the  DAS 
operates.  This  triad  of  systems  and  processes  is  unique  in  that  many  elements  of  risk  and 
uncertainty  are  introduced  by  the  processes  that  the  systems  then  attempt  to  mitigate. 


Figure  4.  DoD  Decision  Support  Systems 

Figure  4  is  a  Venn  diagram  that  shows  the  interrelations  between  the  different 
components  of  the  DoD  Decision  Support  Systems  Triad.  Each  element  interacts  with 
the  other  with  the  Joint  Capabilities  Integration  and  Development  System  (JCIDS)  and 
the  DAS,  both  of  which  seek  to  implement  the  goals  of  the  Planning  Programming, 
Budgeting,  and  Execution  Process.  The  office  responsible  for  oversight  of  each  element 
is  listed  along  with  the  relevant  directives  and  instructions. 

a.  Planning,  Programming,  Budgeting  and  Execution  Process 

This  is  an  annual,  calendar  driven  process.  The  Commander-in-Chief 
formulates  an  overall  National  Security  Strategy  (NSS).  The  Secretary  of  Defense  and 
Joint  Chiefs  of  Staff  then  draft  the  National  Defense  and  National  Military  Strategies, 
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which  are  more  detailed  plans  on  the  way  forward  to  NSS  implementation.  The  Defense 
Budget  is  submitted  to  the  White  House’s  Office  and  Management  Budget  in  January  and 
the  President’s  Budget  delivered  to  Congress  on  the  first  Monday  in  February.  The 
Congressional  budget  process  usually  lasts  from  February  to  September,  and  once  the 
various  authorization  and  appropriation  acts  are  passed,  the  appropriated  funds  to  support 
contracts  will  be  distributed. 

Political  and  budget  risks  originate  in  this  process.  A  new  Commander  in 
Chief  may  want  to  draw  down  the  military  or  a  Secretary  of  Defense  might  attempt  to 
close  down  programs  that  do  not  coincide  with  a  particular  strategic  view.  Legislators 
will  seek  to  provide  benefits  to  their  home  districts  by  awarding  contracts  or  maintaining 
defense,  industrial,  and  research  facilities.  Depending  on  the  political  landscape  during 
the  budget  process,  a  program  may  suddenly  lose  money  or  even  be  terminated. 

b.  Defense  Acquisition  System 

This  system  is  event  driven  in  that  specific  criteria  must  be  met  before  a 
program  can  enter  a  new  phase.  There  are  four  major  milestone  decision  points  in  which 
a  program  can  be  cancelled  if  it  is  not  meeting  cost,  performance,  and  schedule  goals. 

Technology  projects  and  acquisitions  programs  are  categorized  based  on 
its  location  in  the  acquisition  process,  dollar  value,  and  Milestone  Decision  Authority 
(MDA)  special  interest.  Generally,  the  more  expensive  or  important  a  project  or  program 
is,  the  higher  the  level  of  oversight  and  regulation. 

The  major  Acquisition  Categories  (AC  AT)  are  ranked  from  AC  AT  I  to 
ACAT  III.  ACAT  I  projects  and  programs  are  estimated  by  the  USD  (AT&L)  to  require 
a  total  expenditure,  development,  test  and  evaluation  (RDT&E)  of  more  than 
$365  million  based  on  Fiscal  Year  2000  constant  dollars  or,  for  procurement,  of  more 
than  $2.19  billion  in  FY  2000  constant  dollars,  while  total  expenditure  for  ACAT  II 
RDT&E  would  be  expected  to  cost  more  than  $140  million  in  FY  2000  dollars  and  for 
procurement  more  than  $660  million.  ACAT  III  programs  are  smaller  programs  that  do 
not  meet  the  criteria  for  ACAT  II  and  I  programs  (Undersecretary  of  Defense  [AT&L]., 
2008). 
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Two  important  roles  in  the  acquisition  system  are  those  of  the  PM  and 
MDA.  This  thesis  will  concentrate  on  how  both  PMs  and  MDA  attempt  to  mitigate  risk 
and  uncertainty. 

(1)  Program  Manager:  A  PM  is  assigned  to  every  acquisition 
program  and  is  experienced  in  DoD  needs  and  constraints.  They  are  also  responsible  for 
drafting  achievable  and  measurable  annual  plans  that  are  fully  resourced  and  reflect  the 
approved  program  (Undersecretary  of  Defense  [AT&L],  2008).  PMs  report  to  a  Program 
Executive  Officer  (PEO),  which  is  an  official  responsible  for  directing  several  major 
acquisition  programs,  usually  in  the  same  technological  area  or  warfare  field.  For 
example  in  the  USN,  PEO-IWS  is  responsible  for  integrated  warfare  systems  and  PEO- 
C4I  is  responsible  for  communications  and  intelligence  systems. 

The  roles  and  responsibilities  of  a  PM  are  all  encompassing  with 
the  PM  responsible  for  every  facet  of  his  or  her  program.  On  a  day-to-day  basis,  a  PM 
needs  to  demonstrate  a  wide  variety  of  skills  from  technical  expertise,  leadership, 
management,  communication,  and  administrative  skills. 
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Hard  Skills 

1. 

Determine  program  goals,  requirements,  and  specifications 

2. 

Determine  program  scope  and  deliverables 

3. 

Technical  ability 

4. 

Document  program  constraints  that  could  affect  program  completion 

5. 

Document  program  assumptions 

6. 

Define  program  strategy  or  alternative  approaches 

7. 

Quality  assurance 

8. 

Identify  resources  requirements 

9. 

Develop  a  budget 

10. 

Create  a  work  breakdown  structure  (WBS) 

11. 

Develop  a  schedule 

12. 

Develop  a  resource  management  plan 

13. 

Establish  program  controls  comparing  actual  against  planned  performance 

14. 

Develop  program  plan 

15. 

Communicate  program  status 

16. 

Measure  program  performance  to  identify  program  trends  and  variances 

17. 

Implement  corrective  action 

18. 

Implement  change  control 

19. 

Respond  to  risk 

20. 

Conduct  administrative  closure  of  the  program  upon  completion 

Management  /  Leadership  (Soft  Skill)  Competencies 

1. 

Project  leadership 

2. 

Flexibility  to  adapt  and  deal  with  situations  and  manage  expectations 

3. 

Sound  business  judgment 

4. 

Trustworthiness 

5. 

Communication  style  presents  clear  and  unambiguous  information  without  bias 

6. 

Listening  skills 

7. 

Setting  and  managing  expectations 

8. 

Negotiations 

9. 

Issue  and  conflict  resolution 

10. 

Organizational  skills 

11. 

Coaching 

12. 

Facilitation 

13. 

Decision  Making 

14. 

Problem  solving 

15. 

Team  building 

Table  2.  List  of  PM  Competencies  (From  Wood,  2010) 
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Table  2  displays  data  gathered  from  a  survey  of  146  civilian 
industry  managers  that  listed  common  DoD  PM  competencies  (Wood,  2010).  The 
competencies  are  divided  by  20  technical  and  business  or  -hard  skills”  and  15  leadership 
and  management  -soft  skills.”  The  original  intent  of  the  survey  was  to  determine  how 
well  DoD  PMs  are  perfonning  in  certain  skill  areas  though  for  the  purpose  of  this  thesis 
this  list  gives  insight  into  the  diversity  of  skill  sets  needed  by  PMs  and  the  relative  order 
of  importance  of  different  competencies  as  determined  by  their  peers  in  civilian  industry. 

One  compelling  data  point  in  Table  2  is  that  the  competency  of 
responding  to  risk  is  the  only  risk  management  competency  listed  and  the  skill  was  in 
responding  to  risk  instead  of  risk  prevention.  Although  it  could  be  argued  that,  many 
other  skills  such  as  quality  assurance  and  resources  requirements  identification  serve  as 
preventive  measures  against  different  types  of  risk.  The  survey  ranked  35  competencies 
in  order  of  importance  and  responding  to  risk  was  ranked  13th.  Overall,  the  most 
important  competencies  were  determining  program  goals  and  deliverables,  and  the 
trustworthiness  and  leadership  ability  of  the  PM  followed  by  the  administrative  skill  of 
developing  a  budget,  then  team  building.  From  these  rankings,  we  can  detennine  that 
PMs  are  most  interested  in  achieving  a  detailed  knowledge  of  their  programs  and 
developing  leadership  skills,  and  that  their  attitude  towards  does  not  emphasize  risk 
prevention. 

A  GAO  report  highlighted  some  factors  that  differentiate  the 
environment  that  a  DoD  PM  operates  in  compared  to  her  civilian  counterpart.  One  factor 
is  that  the  GAO  found  that  many  programs  began  without  a  business  case,  meaning  that 
there  was  not  a  sufficient  understanding  of  the  technical  issues  and  future  costs  and  time 
needed,  despite  the  JCIDS  requirements  and  the  Material  Solution  Analysis  Phase  of  a 
system’s  life  cycle.  When  confronted  with  problems  a  PM  -eannot  veto  new 
requirements,  control  funding,  or  control  staff’  (U.S.  Government  Accountability  Office, 
2005).  The  result  is  that  PMs  must  constantly  advocate  for  their  programs,  which  is 
another  way  to  describe  the  attempt  to  mitigate  political  risk.  This  fact  plays  a  large  role 
in  what  one  PM  described  in  the  interview  process  as  spending  a  large  portion  of  his  time 
on  protecting  his  budget. 
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(2)  Milestone  Decision  Authority:  Has  overall  responsibility  over 
a  program  and  establishes  mandatory  procedures  for  it.  The  MDA  has  the  authority  to 
approve  entry  of  an  acquisition  program  into  the  next  phase  of  the  acquisition  process  and 
shall  be  accountable  for  cost,  schedule,  and  perfonnance  reporting  to  higher  authority 
(Undersecretary  of  Defense  [AT&L],  2008). 

c.  Joint  Capabilities  Integration  and  Development  System 

Need  driven  and  requirements  based,  JCIDS  serves  to  identify  the 
capabilities  required  by  the  armed  forces  to  support  the  National  Defense  Strategy  and 
ensure  these  capabilities  are  identified  with  their  associated  operational  perfonnance 
criteria  (CJCSI,  2009).  JCIDS  attempts  to  mitigate  risks  that  can  affect  perfonnance 
though  costs  are  also  taken  into  account.  JCIDS  detennines  if  there  is  a  capability  gap 
between  the  National  Defense  Strategy  and  the  order  of  battle  of  the  Armed  Forces  and 
decides  if  non-material  solutions  (change  in  doctrine,  procedure,  regulations,  etc.), 
material  solutions,  or  both,  will  close  the  gap.  Current  weapons  systems  are  screened  to 
see  if  they  can  fill  the  capability  gap  with  minor  changes  and  proposed  programs  are 
reviewed  for  the  technological  feasibility  to  develop  and  deploy  at  a  reasonable  cost. 

2.  Defense  Acquisition  System’s  Mitigation  of  Risk 

To  further  the  goals  of  the  NSS  and  the  National  Defense  and  Military  Strategies 
that  spring  from  it  the  DAS  covers  a  diverse  range  of  areas.  Oversight  and  review, 
contracting,  logistics  and  long  term  sustainment  of  the  system,  technical  evaluations,  and 
financial  management  all  play  a  role  in  the  process  from  detennining  the  problem’s 
solution,  through  to  the  deployment  and  eventual  retirement  of  a  system.  Ultimate 
program  success  is  detennined  by  how  well  a  program  meets  cost,  schedule,  and 
performance  goals. 


a.  Initial  Capabilities  Document 

An  Initial  Capabilities  Document  (ICD)  describes  broad,  time  phased, 
operational  goals  and  needed  capabilities  from  a  warfare  community,  or  jointly  by  many 

communities  within  the  DoD.  If  the  ICD  identifies  a  material  solution  then  a  MDA  will 
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be  assigned  to  decide  if  the  ICD  contains  sufficient  information  to  proceed  to  the  first 
phase.  At  this  stage,  the  first  attempts  to  mitigate  technical  risk  start  with  the 
identification  of  promising  foreign  and  domestic  technical  sources,  though  a  price  is  paid 
in  time  by  PMs  for  the  mandatory  consideration  of  technologies  developed  under  the 
Small  Business  Innovation  Research  program  (Undersecretary  of  Defense  [AT&L], 
2008).  This  time  cost  does  not  yet  present  schedule  risks,  as  the  program  has  not  yet 
officially  entered  the  acquisition  system  at  this  stage. 

Future  integration  risk  is  mitigated  by  the  requirement  to  abide  by  the 
DoD  Enterprise  Architecture  (DoD  Directive  8000.01)  during  infonnation  architecture 
development  and  that  all  standards  used  for  form  the  technical  view  of  integrated 
architectures  are  contained  in  the  approved  version  of  the  DoD  IT  Standards  Registry. 

b.  Material  Solution  Analysis  Phase 

This  phase  beings  with  a  Material  Development  Decision  review  and  is 
concerned  with  the  study  of  any  alternatives  available  besides  the  solution  articulated  in 
the  ICD.  This  serves  to  avoid  costs  spent  on  developing  redundant  systems  and 
technologies.  If  reasonable  alternatives  are  not  found  then  the  material  solution 
contained  in  the  ICD  shall  be  reviewed  for  measures  of  effectiveness,  cost,  schedule, 
concepts  of  operations,  and  overall  risk.  The  avoidance  of  program  risk  begins  with  the 
Analysis  of  Alternatives  study  assessing  critical  technology  elements  (CTE)  of  the 
material  solution  to  include  technology  maturity,  integration  risk,  and  manufacturing 
feasibility  (Undersecretary  of  Defense  [AT&L],  2008). 

Future  cost  savings  are  sought  by  emphasizing  innovation  and  competition 
when  seeking  the  best  possible  system  solution  and  this  emphasis  on  competition  has 
become  a  principle  of  Navy  OA.  Future  budget  and  integration  risks  are  mitigated  by 
the  use  of  COTS  solutions  though  this  tends  to  increase  security  risks  as  exploits  on 
commercial  systems  are  more  well  know  than  on  systems  developed  in  house  by  the  DoD 
and  not  exposed  to  the  public. 
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c.  Technology  Development  Phase 

Before  beginning  this  phase,  a  program  has  to  pass  through  Milestone  A, 
which  consists  of  the  MDA  reviewing  the  proposed  material  solution  and  the  Technology 
Development  Strategy  drafted  by  the  PM.  After  MDA  approval,  the  Technology 
Development  Phase  begins.  From  the  DoDI  5000.02: 

The  purpose  of  this  phase  is  to  reduce  technology  risk,  determine,  and 
mature  the  appropriate  set  of  technologies  to  be  integrated  into  a  full 
system,  and  to  demonstrate  CTEs  on  prototypes. 

This  phase  takes  the  theoretical  ICD  and  the  findings  of  the  Material 
Solutions  phase  and  starts  physical  testing  and  hands  on  development  of  new 
technologies.  Future  budget  risks  and  threats  to  cost  metrics  are  addressed  in  this  phase. 
If  the  PM  estimates  that  the  cost  estimate  used  by  the  MDA  that  was  based  on  the 
Milestone  A  certification  increases  by  over  25  percent  then  he  shall  notify  the  MDA  who 
detennines  if  the  expense  of  system  development  can  be  justified  in  light  of  the  priority 
level  assigned  by  the  Joint  Requirements  Oversight  Council  (JROC). 

Once  a  detennination  is  made  that  the  program  is  worth  the  expense,  and 
the  technical  and  manufacturing  processes  have  been  assessed  and  demonstrated  in  a 
relevant  environment,  the  program  can  exit  the  Technology  Development  Phase. 
Manufacturing  risks  will  have  been  identified,  and  testing  should  confirm  that  the 
system  could  be  developed  within  a  short  timeframe,  which  is  defined  as  less  than  five 
years  for  a  weapons  system  (Undersecretary  of  Defense  [AT&L],  2008). 

Before  Milestone  B  a  PM  shall  conduct  a  Preliminary  Design  Review 
(PDR).  This  report  is  given  to  the  MDA  and  lists  any  trade-offs  in  requirements  based 
upon  an  assessment  of  cost,  schedule,  and  performance  risks.  The  MDA  then  decides 
if  any  remidial  action  is  necessary  before  proceding  to  the  next  phase. 

d.  Engineering  and  Manufacturing  Development  Phase 

This  phase  begins  at  Milestone  B  in  which  the  Acquisition  Strategy  and 
Acquisition  Program  Baseline  is  approved,  addressing  Integrated  System  Design,  and  the 


29 


System  Capability  and  Manufacturing  Process  Demonstration.  Key  Performance 
Parameters  (KPP)  are  identified  and  approved. 


Integration  and  system  level  risks  are  mitigated  during  the  Integrated 
System  Design  as  system  functionality  and  interfaces  are  defined,  and  a  hardware  and 
software  design  produced.  The  establishment  of  a  product  baseline  for  all  configuration 
items  prevents  integration  risk. 

A  system  level  Critical  Design  Review  (CDR)  assesses  the  design 
maturity  and  feasibility  of  a  program.  Metrics  include  percentages  of  hardware  and 
software  products  built  to  specifications,  numbers  of  drawings  completed  assessments  of 
environmental,  safety  and  occupational  health  risks,  and  the  maturity  of  critical 
manufacturing  processes.  The  MDA  then  conducts  a  Post-CDR  Assessment  and 
detennines  if  the  program  can  exit  the  Integrated  System  Design  part  of  this  phase.  With 
MDA  approval,  the  program  moves  to  a  System  Capability  and  Manufacturing  Process 
Demonstration.  This  is  where  performance  risks  are  mitigated  by  detennining  if  the 
system  can  operate  in  accordance  with  the  KPPs  and  schedule  risks  by  demonstrating 
that  the  manufacturing  processes  can  support  system  production  goals. 

e.  Production  and  Deployment  Phase 

At  Milestone  C,  the  MDA  decides  if  the  DoD  will  be  committed  to 
production.  Major  systems  will  begin  at  a  low  rate  of  production,  which  helps  prevent 
against  program,  technical,  and  performance  risks.  These  risks  are  avoided  by  the 
identification  of  any  issues  arising  during  the  actual  manufacturing  of  system  components 
and  the  testing  of  these  products  instead  of  issues  developing  during  a  full  rate  of 
production.  Low  rates  of  production  are  not  applicable  to  automated  infonnation  systems 
or  software  intensive  systems  without  developmental  hardware  but  this  decision  is  left  up 
to  the  PM  (Undersecretary  of  Defense  [AT&L],  2008).  A  Full  Rate  Production  Decision 
Review  is  required  before  the  program  can  pass  into  full  rate  production. 
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/.  Operations  and  Support  Phase 

This  phase  begins  with  delivery  of  the  system  to  the  operational  forces 
with  a  goal  of  meeting  operational  and  performance  requirements  in  the  most  cost 
effective  manner  over  the  system’s  life  cycle.  PM’s  use  Perfonnance  Based  Life  Cycle 
Product  Support  planning,  development,  implementation,  and  management.  The  goal  is 
to  keep  the  system  reliable  while  keeping  costs  down. 

This  phase  also  includes  the  disposal  portion  in  which  a  system  is  disposed 
of  in  accordance  with  legal  and  regulatory  requirements. 

g.  Relative  Weight  of  Mitigation  for  Various  Risks  in  the  Defense 
Acquisitions  System 

Cost,  schedule,  and  perfonnance  and  the  mitigation  of  risks  that  can  affect 
these  areas  are  the  prime  concerns  of  every  program’s  PM  and  PEO,  though  a  bird’s  eye 
view  of  the  DAS  as  revealed  in  the  preceding  paragraphs  of  this  chapter  reveals  an 
emphasis  on  mitigating  integration  and  technical  risks.  Integration  risks  are  a  subset  of 
performance  risks  though  the  attempt  to  achieve  greater  levels  of  integration  in  a  system 
can  also  affect  the  cost  and  schedule  of  a  program.  The  emphasis  on  integration  risk  is 
understandable  due  to  the  change  in  doctrine  of  the  U.S.  Armed  Forces  to  Joint  Warfare 
after  the  Goldwater-Nichols  DoD  Reorganization  Act  of  1986  and  other  Federal 
legislation  such  as  the  Clinger-Cohen  Act  of  1996  that  mandates  the  way  information 
technology  is  acquired. 

The  mitigation  of  technical  risks  forms  20  percent  of  the  acquisition  cycle 
as  it  is  the  goal  of  the  Technology  Development  Phase  while  perfonnance,  and  the 
mitigation  of  risks  associated  with  them  form  the  basis  for  the  remaining  three  phases 
after  Milestone  B.  Budget  risks  are  mostly  addressed  in  the  first  two  phases.  Budget, 
performance,  and  technical  risks  have  an  equal  weight  when  viewed  in  the  system  as  a 
whole,  but  not  on  the  scale  of  integration  risks,  which  is  emphasized  throughout  the 
whole  cycle. 

Manufacturing  risks  closely  align  with  schedule  risk,  though  issues  that 
arise  in  the  manufacturing  process  that  can  influence  cost  and  performance  are  mostly 
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emphasized  after  the  system  design  has  been  proven.  Most  of  the  emphasis  on  mitigating 
manufacturing  risks  come  after  the  Post-CDR  in  the  Engineering  and  Manufacturing 
Development  Phase  and  never  approach  the  level  of  emphasis  given  to  other  areas  of  risk 
mitigation. 

When  reading  Operation  of  the  Defense  Acquisition  System,  DoDI 
5000.02  with  an  eye  towards  the  risks  emphasized  in  the  instruction  a  slightly  different 
picture  appears.  Technical  risks  are  mentioned  more  often  than  any  other  type  of  risk 
while  integration  risk  is  mentioned  less  than  other  types.  Table  2  lists  the  various  risks 
mentioned  in  DoDI  5000.2  and  the  frequency  in  which  they  appear  throughout  the 
instruction  in  order  from  most  frequent  to  least.  When  counting  the  frequency  of  a  type 
of  risk  the  table  ignores  multiple  mentions  of  the  same  type  of  risk  in  the  same  paragraph. 
For  example,  paragraph  9  of  page  25  explains  the  selection  of  a  contract  type  at 
Milestone  B  and  mentions  program  risk  six  times,  though  in  Table  2  it  is  only  counted 
once,  as  it  is  not  mentioned  anywhere  else  in  the  instruction. 


Risk 

Mentioned 

Risk 

Mentioned 

Risk 

Mentioned 

TECHNICAL 

10 

PERFORMANCE 

6 

ENVIRONMENT 

SAFETY  & 

OCCUPATIONAL 

HEALTH 

5 

COST 

5 

SCHEDULE 

4 

MANUFACTURING 

3 

OPERATIONAL 

2 

INTEGRATION 

2 

ENTERPRISE 

2 

ARCHITECTURE 

SYSTEM 

LEVEL 

2 

MATERIAL 

1 

PROGRAM 

1 

Table  3.  Risks  Mentioned  in  DoDI  5000.02 


One  explanation  on  why  an  overview  of  the  DAS  highlights  integration 

risk  but  fails  to  mention  it  as  often  in  the  instruction  is  that  many  mentions  of  technical 

risk  consist  of  mitigating  risks  associated  with  the  integration  of  different  technologies. 

Additionally  integration  risk  can  be  viewed  as  closely  related  to  system  level  risk. 
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III.  RISK  AND  UNCERTAINTY  AS  PERCEIVED  BY 
ACQUISITION  PROFESSIONALS 

To  investigate  the  questions  posed  by  this  thesis,  a  series  of  primary  interviews 
was  conducted  from  April  to  June  2011  with  ten  acquisition  professionals.  Interviews 
were  conducted  in  person  and  over  the  telephone  with  subjects  having  extensive 
experience  serving  either  as  PMs  or  program  Contract  Management  for  various  programs 
in  all  four  services  of  the  DoD.  Some  are  retired  and  some  are  teaching  DAS  subjects  at 
the  Graduate  School  of  Business  and  Public  Policy  at  the  Naval  Postgraduate  School. 
The  other  interview  subjects  are  currently  working  in  various  positions  at  PEO-IWS  and 
PEO-LMW.  Many  of  the  interviews  were  free-flowing  discussions  rather  than  specific 
question  and  answer  sessions,  revealing  some  strong  commonalities  and  consensus  views. 

This  chapter  begins  with  subjects  areas  in  which  all  interviewees  devoted  a 
significant  portion  of  the  conversation  and  which  all  subjects  were  almost  unanimous  in 
their  opinions.  The  three  critical  subject  areas  for  acquisition  professionals  are  budget 
uncertainty,  program  risk,  uncertainty,  and  decreasing  returns  on  increasing  assets.  These 
subjects  came  up  in  discussions  without  the  researcher’s  bias  with  a  prepared  set  of 
questions.  The  second  half  of  this  chapter  explores  the  interview  subject’s  opinions 
concerning  the  thesis  research  questions. 

A.  BUDGET  UNCERTAINTY 


“I  spent  over  a  quarter  of  my  time  either  protecting  my  money  or  stealing  someone  else  "s 

money  ”  -  DAS  Program  Manager 


The  above  quote  by  a  PM  describing  his  experiences  process  succinctly 
summarizes  the  common  concern  over  budget  uncertainty  revealed  during  the  interviews. 
Although  all  interviewees  gave  equal  weight  to  cost,  schedule,  and  performance  issues 
when  discussing  their  programs  and  the  acquisition  system  in  general,  it  was  the  ever 
present  threat  of  budget  reductions  and  program  cancellations  casting  a  constant  shadow 
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of  uncertainty.  Mitigating  budget  risk  is  specifically  not  mentioned  in  acquisition 
directives  and  instructions  and  all  interview  subjects  seemed  to  give  an  equal  weight  to 
cost,  schedule,  and  perfonnance  but  the  primary  interviews  confirmed  budget  risks  as  a 
major  area  of  uncertainty. 

Budget  uncertainty  cannot  be  blamed  on  a  lack  of  attention  to  costs  in  the 
acquisition  cycle.  For  example,  a  good  portion  of  the  Materiel  Solution  Analysis  Phase 
covers  program  feasibility  and  estimation  of  life  cycle  costs  while  the  PDR  before 
Milestone  B  lists  the  trade-offs  in  functionality  required  to  keep  cost  risk  under  control. 
The  interview  consensus  was  that  political  risk  was  a  common  cause  of  much  budget 
uncertainty. 

Political  risk  emerges  from  the  Planning,  Programming,  Budgeting  and  Execution 
Process;  originating  anywhere  from  a  significant  change  in  a  new  Administration’s  NSS 
to  recent  election  results  eliminating  legislative  support  for  a  program  to  a  period  of 
budget  cuts  and  austerity.  A  typical  manifestation  of  political  risk  was  a  Friday  afternoon 
phone  call  from  the  PEO  asking  for  a  Plan  of  Action  and  Milestone  by  Monday  morning 
on  how  cost,  schedule,  and  performance  would  be  affected  by  budget  reclamation  of 
money  as  changes  in  the  political  landscape  slowly  worked  its  way  down  to  the  program 
level.  Many  interview  subjects  also  agreed  with  a  GAO  report  stating  that  many  PMs 
believe  that  a  lack  of  a  strategic  investment  vision  leads  the  DoD  to  -start  more  programs 
than  it  can  afford  and  not  prioritize  them  for  funding  purposes”  (U.S.  Government 
Accountability  Office,  2005).  Lack  of  prioritization  eventually  leads  to  a  mad  scramble 
for  money  and  Friday  afternoon  data  calls  for  reclamation  of  funds  when  the  DoD  faces 
budgetary  pressure. 

The  best  strategy  to  mitigate  budget  risk  and  reduce  the  impact  of  the  Friday 

afternoon  phone  call  was  to  expect  one’s  budget  to  come  under  review  for  cuts  and  have 

plans  and  templates  for  various  levels  of  budget  cuts  already  drafted  and  readily  on  hand. 

In  fact,  one  interview  subject  discounted  any  uncertainty  when  it  came  to  his  budget  as  it 

was  a  -certain  uncertain  that  I  could  count  on  folks  coming  to  take  my  money.”  In  order 

to  achieve  this  level  of  preparation  it  is  not  surprising  that  budget  issues  can  take  up  80 

percent  of  a  PM’s  time.  -Stealing”  another  program’s  money  is  also  time  consuming  but 
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can  pay  dividends  for  a  PM  devoting  the  time  necessary  to  follow  the  progress  of  other 
programs  and  capitalizes  on  opportunities  to  move  money  into  his  program  if  another 
program  is  terminated  or  money  is  reallocated  elsewhere.  One  interviewee  used  the 
analogy  of  what  happens  in  DAS  when  a  larger  program  is  terminated  to  a  large  shark 
being  overwhelmed  by  a  swann  of  smaller  ACAT  level  barracuda. 

In  the  end,  there  is  not  much  a  PM  can  due  to  mitigate  against  political  causes  of 
budget  risk  besides  knowing  what  levels  of  budget  cutting  pain  programs  can  survive  and 
still  meet  cost,  schedule,  and  performance  goals.  In  addition,  keeping  a  level  of 
awareness  allows  a  PM  to  know  when  new  sources  of  money  may  be  available. 

B.  KEEPING  IT  ALL  TOGETHER,  PROGRAM  RISK  AND  UNCERTAINTY 

“Uncertainty  equals  many  different  independent  parts  Graduate  School  of  Business  & 

Public  Policy  professor 

When  discussing  various  ways  risk  and  uncertainty  has  been  defined  in  different 
industries  and  technical  fields,  a  veteran  acquisition  professional  offered  the  above  quote. 
It  was  not  his  intention  to  mention  program  risks  in  DAS  but  he  inadvertently  uncovered 
the  most  mentioned  risk  type  during  the  interview  process.  Without  specifically  asking 
interviewees  what  was  the  most  problematic  risks,  70  percent  emphasized  program  risks 
over  all  others.  All  interview  subjects,  with  the  exception  of  one,  spent  a  significant 
portion  of  the  interviews  discussing  program  risks,  which  the  same  acquisition 
professional  that  gave  the  definition  of  uncertainty  above  defined  as  programmatic  risk, 
or  a  combination  of  requirement  and  budget  risks  and  a  companion  to  cost,  schedule,  and 
performance  risks.  For  the  purposes  of  this  thesis,  a  program  risk  is  any  risk  threatening 
a  project  or  program  meeting  its  cost,  schedule,  and  perfonnance  goals.  These  risks 
include  those  beyond  requirements  and  include  risks  emerging  from  areas  such  as 
composition  of  the  acquisition  workforce;  complexity  of  system  due  to  an  abundance  of 
rules  and  regulation;  and  even  rules  and  regulations  contradicting  each  other.  These  risks 
are  discussed  further  in  this  section. 

One  recurring  theme  during  the  interviews,  in  the  area  of  performance  risks,  was 
the  problem  that  different  groups  and  persons  involved  in  DAS  tended  to  -talk  by  each 
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other”  and  battled  in  a  -el ash  of  perspectives”  caused  by  conflicting  rules  between  the 
functional  areas  at  even  the  Integrated  Product  Team  (IPT)  level.  A  common  cause  of 
friction  are  the  different  priorities  and  motivations  of  a  PM  and  a  contracting  officer.  A 
contracting  officer  is  assigned  to  a  program  not  only  to  serve  as  the  program’s  subject 
matter  expert  on  contracting  but  is  also  required  to  ensure  that  the  program  abides  by  the 
will  of  the  Executive  and  Legislative  Branches  as  expressed  in  such  legislation  as  the 
Truth  in  Negotiations  Act,  Small  Business  Act  and  the  Anti-Deficiency  Act. 

Frictions  also  arise  when  a  PM  wants  to  avoid  cost  and  schedule  risk  by  relying 
upon  a  large  defense  contractor  and  possibly  overlooking  portions  of  the  Small  Business 
Act  to  avoid  risks  associated  with  a  smaller  company  with  limited  production  capability, 
or  more  likely  to  go  bankrupt  than  a  larger  one.  Many  PMs  generally  do  not  receive 
much  in  the  way  of  contracting  training  and  in  many  programs;  the  PM  is  a  senior  field 
grade  officer  that  may  outrank  the  contracting  officer  by  three  or  four  pay  grades.  A  PM 
may  pressure  a  junior  contracting  officer  to  approve  a  contract  that  might  not  pass  muster 
with  various  laws  and  regulations.  The  PM  is  under  intense  career  pressure  to  meet 
sometimes  unrealistic  cost  and  schedule  goals  and  many  times  is  not  responsible  for  the 
setting  of  these  goals  as  the  original  PM  transferred  after  the  goals  were  established.  The 
PM  may  resort  to  a  -mission  first”  approach  common  in  the  military  and  probably  a 
factor  in  the  PM’s  career  success  so  far.  If  the  PM  is  in  an  untenable  situation  and 
decides  not  to  bend  the  rules  to  meet  unrealistic  goals,  the  PM  may  very  well  ruin  his 
career.  A  junior  contracting  officer  has  the  same  choice  when  pressured  by  a  PM  to  bend 
the  rules.  The  contracting  officer  can  refuse  to  bend  the  rules  and  risk  career  suicide  or 
go  along  with  the  PM  and  hope  that  the  rule  bending  does  not  become  an  issue,  especially 
if  the  trend  is  to  do  what  needs  to  be  done.  A  contracting  officer  may  very  well  go  along 
with  a  PM  in  approving  a  contract  that,  for  example,  breaks  the  Anti-Deficiency  Act, 
especially  since  no  one  has  ever  been  indicted  for  violating  it  (Arnold,  2009). 

The  myriad  of  risks  confronting  a  program  is  illustrated  by  the  above  example. 
Frictions  exist  between  functional  areas  of  contracting  and  program  management,  along 
with  risks  introduced  by  the  lack  of  training,  and  the  personnel  system. 


36 


Besides  the  inherent  friction  between  the  different  functional  areas  in  the  DAS, 
there  is  also  a  risk  of  linguistic  discontinuity.  This  type  of  risk  was  described  in  a  paper 
exploring  a  problem  inherent  in  software  engineering  in  which  many  different  linguistic 
styles  are  used  to  create  software,  often  leading  to  confusion  and  rework  within  a  project 
(Riehle,  2006)  and  can  include  language  structure,  semantics,  grammar,  and  syntax.  An 
example  is  given  in  the  paper  of  a  software  project  PM  that  planned  to  use  an  object 
oriented  approach  to  the  software  design.  A  group  outside  of  the  PMs  control  used  a 
popular  computer  aided  software  engineering  (CASE)  tool  that  supported  a  structured 
design  rather  than  an  object  oriented  approach  immediately  introducing  linguistic 
discontinuity  between  the  CASE  tool  and  the  rest  of  the  code  necessitating  extra  work  in 
making  the  dissimilar  tools  compatible. 

Every  software  project  will  experience  linguistic  discontinuity  of  one  fonn  or 
another  as  the  end  user’s  and  operator’s  language  is  not  as  technical  as  the  software 
programmers  and  designers.  The  phenomena  of  linguistic  discontinuity  is  not  just  a 
software  engineering  risk  but  is  apparent  in  any  project,  both  in  the  DoD  and  in  the 
business  world.  If  software  coders  can  misunderstand  one  another  when  both  have  the 
same  skill  sets  and  end  goal  the  risk  of  linguistic  discontinuity  between  the  different 
functional  areas  of  the  DAS  is  much  more  intense.  With  linguistic  discontinuity  residing 
everywhere  in  the  DAS  from  within  an  IPT,  to  between  the  functional  specialists  in  the 
DAS  such  as  the  PM,  Contracting  Officer,  and  logistics,  it  is  little  wonder  that  program 
risk  is  a  main  concern  of  acquisition  professionals. 

A  major  program  risk  is  that  a  system  ready  for  deployment  turns  out  not  to  meet 
end  user  requirements.  The  fear  of  either  misunderstanding  or  overlooking  vital  end  user 
requirements  was  a  constant  topic  during  the  interview  process  for  this  thesis.  PM’s 
would  stress  the  need  to  -define  requirements  with  the  end  users  up  front”  and  to  engage 
them  constantly  during  the  development  cycle.  The  warning  that  a  PM  should  -own  your 
own  requirements”  refers  to  the  common  occurrence  of  the  government  providing  general 
requirements  and  specifications  to  the  contractor,  not  following  up  on  the  contractor’s 
work,  only  to  be  presented  with  an  inadequate  product.  An  example  of  an  unforeseen 
user  requirement  that  neither  the  end  users  or  the  Integrated  Product  Team  thought  of  was 
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given  during  an  interview  with  an  acquisition  professional  involved  with  upgrading  the 
U.S.  Military  Entrance  Processing  Command’s  network.  The  goal  of  the  upgrade  was  to 
allow  the  network  to  be  able  to  receive  and  store  information  from  the  end  user’s 
networks,  in  this  case  from  the  various  recruiting  stations  around  the  country.  After  a 
three-year  development  cycle,  the  vendor  released  their  solution  only  to  find  that  a  data 
entry  field  for  each  recruit’s  shoe  size  was  missing.  The  end  users  needed  that 
information  to  send  on  to  the  various  boot  camps  so  enough  shoes  of  the  proper  sizes 
could  be  on  hand.  This  omission,  while  minor,  was  one  of  the  many  cases  of  end  user 
requirements  not  being  met  that  caused  a  delay  in  the  program. 

Not  all  program  risks  can  be  attributed  to  misunderstandings  or  failure  of  the 
contractors  to  develop  systems  to  government  specifications  as  another  constant  risk  to 
all  PMs  as  told  to  the  researcher  during  the  interview  process  was  scope  creep.  Scope 
creep  is  a  risk  that  one  encounters  when  trying  to  mitigate  against  the  risks  of 
misunderstanding  user  requirements.  While  all  PMs  agreed  that  constant  communication 
and  engagement  between  all  parties  was  a  key  to  a  successful  program,  the  tendency  was 
for  the  government  or  the  end  users  to  add  more  requirements  beyond  the  initial 
capabilities  document.  Through  the  complexity  added  by  new  requirements  and  the  fact 
that  -vendors  don’t  do  things  for  free”  the  program  risk  increases. 

The  interview  subjects  mentioned  other  areas  that  increase  program  risk  to 
include  a  lack  of  accountability  due  to  constant  personnel  rotation  and  senior  leadership 
that  -doesn’t  know  what  they  don’t  know”  and  believe  that  -brute  force”  leadership  will 
work.  A  personnel  problem  mentioned  in  at  least  a  third  of  the  interviews  was  that 
despite  a  DoDI  5000.02  requirement  that  PMs  serve  in  an  AC  AT  II  level  program  for  at 
least  three  years  (four  years  for  ACAT  I),  there  are  many  cases  in  which  a  PM  does  not 
stay  that  long  due  to  a  promotion,  retirement,  etc.  A  GAO  report  revealed  that  in 
comparison  with  the  DAS  civilian  companies  usually  kept  the  same  PM  throughout  the 
life  span  of  a  product,  and  this  was  the  main  means  of  ensuring  accountability  (U.S. 
Government  Accountability  Office,  2005)  while  DoD  PMs  were  rarely  held  accountable. 
Through  the  course  of  the  interview  process  the  researcher  discovered  a  general 
agreement  that  the  requirements  placed  upon  PMs  in  the  DAS  far  exceed  the  flexibility 
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and  decision  making  power  allowed  to  achieve  them  so  that  -the  system  falls  so  far  short 
of  the  mark  that  it  would  be  almost  criminally  unfair  to  hold  them  (PMs)  responsible  for 
its  failures”  (Shoop,  2005) 

Another  personnel  issue  commonly  mentioned  was  the  nature  of  the  Government 
Service  (GS)  personnel  system,  which  emphasizes  time  based  service  over  a  merit  based 
promotion  or  retention  system.  Personnel  issues  will  continue  to  be  an  issue  in  the  DAS 
as  a  Government  Accounting  Office  (GAO)  report  mentioned  the  DoD’s  acquisition 
workforce  plan,  in  which  the  DoD  identified  the  need  to  increase  the  size  of  the 
acquisition  workforce  but  found  that  the  DoD  had  not  yet  assessed  the  skills  and 
competencies  of  the  workforce,  or  identified  either  the  desired  end  state  of  the  acquisition 
workforce  or  the  funding  required  (Farrell  &  Hutton,  2011).  The  ongoing  budget  issues 
may  stop  any  planned  increased  in  the  acquisition  workforce  but  the  lack  of  prior 
planning  given  to  how  the  DoD  acquisition  workforce  will  be  structured  in  the  future 
makes  it  extremely  unlikely  there  will  be  any  major  changes  in  the  personnel  system. 

Problems  associated  with  program  risk  are  wide  ranging  and  many  are  outside  of 
the  control  of  the  DoD.  The  consensus  of  all  interview  subjects  can  be  summarized  in  a 
statement  given  by  one  that  -we  are  stuck  with  the  hand  we  are  dealt  with  when  it  comes 
to  personnel.”  Various  suggestions  for  mitigation  strategies  did  come  out  of  the 
interviews.  On  way  to  mitigate  program  risk  was  more  training  and  education,  especially 
training  tailored  to  explaining  other  functional  areas  of  the  DAS  to  include  the 
requirements,  laws,  and  regulations  unique  to  a  functional  area.  Most  interview  subjects 
agreed  that  a  PM  was  at  the  mercy  of  the  government  and  DoD  personnel  and  acquisition 
workforce  policies  and  could  not  provide  much  practical  advice  on  how  to  mitigate  these 
issues.  All  agreed  that  a  delay  in  ramifications  for  bad  program  decisions  was  an  issue. 
For  example,  a  Program  Manager  may  favor  awarding  a  contract  to  the  lowest  bidder  in 
an  attempt  to  keep  costs  low  and  at  the  expense  of  future  integration  or  technical  risk.  If 
the  PM  is  slated  to  transfer  before  the  integration  risks  can  manifest  themselves  he  may 
be  tempted  to  award  the  contract  to  the  low  bidder. 
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C.  DECREASING  RETURNS  ON  INCREASING  INVESTMENTS 


“The  bureaucracy  is  expanding  to  meet  the  needs  of  the  expanding  bureaucracy.  ” 

-Oscar  Wilde 

When  discussing  schedule  risks  a  commonality  in  the  interview  responses  was  the 
bureaucratic  nature  of  the  DAS  and  a  consensus  that  the  overall  trend  was  a  continued 
movement  towards  more  oversight  and  regulation  as  opposed  to  allowing  PMs  to  exercise 
much  in  the  nature  of  flexibility  and  initiative.  The  accumulation  of  rules,  regulations, 
legislation,  and  procedures  that  have  built  up  over  the  years  and  forms  the  basis  for  the 
DAS  is  the  result  of  various  attempts  to  mitigate  risks  and  uncertainties.  With  every 
mandatory  process,  test,  paperwork,  and  validation  an  increase  in  costs  and  time  can  be 
expected.  The  trade  off  is  that  the  increase  in  costs  and  time  are  worth  the  expense  in 
order  to  assure  successful  performance  and  a  safe,  secure,  system  delivered  to  the 
operating  forces  and  upon  which  lives  may  depend. 

After  the  interview  process,  the  conclusion  of  the  researcher  is  that  PMs  view  the 
bureaucratic  nature  of  the  DAS  as  a  permanent  feature  of  the  environment  and  is 
something  that  has  must  be  endured  as  opposed  to  trying  to  introduce  efficiencies  in  the 
system  or  fight  for  more  flexibility  and  latitude  in  day-to-day  management  of  their 
programs.  This  fatal  outlook  is  driven  by  the  main  difference  between  the  DAS  and 
civilian  industry.  A  business  is  more  apt  to  change  its  internal  business  rules  and 
personnel  policies  if  they  hinder  the  profitability  of  a  company.  Additionally,  the 
company  has  full  control  over  its  own  policies  and  only  have  to  worry  about  as  many 
Federal  regulations  that  pertain  to  its  business  and  work  force  as  compared  to  the  DAS. 
On  the  other  hand,  the  DAS  is  once  removed  from  the  source  of  funding  and  many  of  the 
rules  and  regulations  that  originate  in  the  Planning,  Programming,  Budgeting,  and 
Execution  Process.  Much  of  the  money  that  flows  to  the  DAS  is  apportioned  based  on 
political  reasons  such  as  the  award  of  a  contract  in  the  home  district  of  a  legislature  or 
comes  with  strings  attached  such  as  pots  of  money  only  accessible  if  the  program  awards 
the  contract  to  a  small  or  minority  owned  business.  Personnel  policies  are  driven  either 
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by  the  DoD  regulations  for  military  personnel  or  GS  regulations.  A  PM  generally  has 
less  flexibility  in  financial  and  personnel  decisions  as  his  civilian  counterpart. 

The  result  is  a  mountain  of  tasks,  requirements,  administrative  requirements, 
reports,  and  paperwork  that  has  slowly  built  up  over  the  years  in  response  to  a  real  need 
or  a  political  impulse.  Taken  individually  each  task  associated  with  the  bureaucratic 
process  is  worthwhile  and  most  would  agree  that  the  cost  and  effort  would  be  worth  it. 
Over  time,  the  cumulative  accumulation  of  controls  has  added  such  time  and  money  costs 
on  daily  operations  that  any  new  imitative  comes  at  every  greater  expense  of  now  limited 
time  and  money  for  smaller  and  smaller  real  gains  in  efficiencies  or  social  good  for 
politically  inspired  requirements. 

D.  RESPONSE  TO  RESEARCH  QUESTIONS 

1.  Risks  at  Different  Career  Stages 

This  question  was  asked  in  order  to  detennine  the  categories  of  risk  important  to 
acquisition  professionals  and  PMs  in  particular.  Differences  in  the  amount  of  risk  at 
different  stages  of  one’s  career  could  indicate  many  things.  Very  high  amounts  of  risk  at 
the  beginning  of  one’s  acquisition  career  could  point  to  the  fact  that  the  DAS’s  operating 
principles  and  techniques  was  unique  and  that  those  with  experience  in  other  branches  of 
the  DoD  and  government  could  not  fall  back  on  prior  work  experience  to  help  them  in 
their  new  career.  High  risk  at  the  beginning  of  a  career  could  also  point  to  inadequate 
acquisition  workforce  accession  programs  and  training  or  may  also  point  to  a  great 
disconnect  between  the  operational  forces  and  the  acquisition  system  that  is  supposed  to 
support  them.  Increased  risks  at  the  mid-point  of  a  career  and  higher  risks  for  well 
seasoned  acquisition  professionals  may  point  to  a  system  that  would  assign  riskier 
programs  to  the  more  those  with  more  experience  or  to  a  system  where  high  profile 
programs  are  subject  to  more  interference  and  uncertainty.  Another  possibility  would  be 
that  there  might  be  programs  and  projects  that  were  virtually  risk  free  and  that 
inexperienced  PM  could  be  assigned  to  them. 

Different  types  of  risks  at  different  stages  of  an  acquisition  professionals  career 
would  reveal  either  a  segregated  system  in  which  there  were  many  programs  and  projects 

41 


that  either  had  different  business  rules  and  regulations  applied  to  them.  Discovery  of 
different  types  of  risk  at  different  stages  of  a  career  might  also  point  to  the  types  of  risk 
that  the  system  would  be  more  or  less  willing  to  endure.  For  instance,  inexperienced 
PMs  might  be  assigned  to  programs  where  the  DoD  was  willing  to  chance  cost  overruns 
and  more  experienced  professionals  would  be  assigned  to  programs  with  an  ambitious 
schedule. 

Amongst  all  interview  subjects,  the  opinion  was  unanimous  that  besides  the  risk 
common  to  all  professions  that  a  newcomer,  no  matter  how  well  trained,  faces,  there  was 
no  real  difference  in  either  the  amount  or  type  of  risks  inherent  in  different  stages  of  an 
acquisition  professional’s  career.  The  researcher’s  opinion  is  that  insignificant  variations 
in  the  amount  and  type  of  risk  over  a  typical  career  indicates  that  the  DAS  applies  the 
same  business  rules  to  all  programs  and  projects  and  that  all  programs  and  projects  are 
handled  in  more  or  less  the  same  way,  no  matter  the  ACAT  level.  The  system  benefits 
from  well  defined  rules  and  that  transitioning  into  the  acquisition  workforce  is  not 
problematic  though  the  trade  off  is  that  the  constant  level  and  type  of  risk  leaves  less 
room  for  individual  initiative  and  an  avoidance  of  seeking  anything  but  a  traditional 
solution  to  a  problem. 

2.  Risks  beyond  Cost,  Schedule,  and  Performance 

Enquiring  after  risks  beyond  cost,  schedule,  and  perfonnance  was  an  attempt  to 
find  if  any  unique  risks  in  the  DAS  or  if  a  PM  had  knowledge  of  a  successful  strategy  to 
mitigate  against  various  types  of  risk  that  was  not  well  known. 

Unsurprisingly,  all  interview  subjects  agreed  upon  cost,  schedule,  and 
performance  risks  as  their  prime  concern.  The  first  half  of  this  chapter  describes  program 
risk;  described  as  a  mix  of  requirement  and  budget  risks,  and  the  well  known  political 
risks  and  feared  budget  risks  that  all  interview  subjects  agreed  were  important.  During 
the  interviews  many  types  of  risks  such  as  design  and  component  design  risks  and 
standards  selection  risks  were  discussed  but  all  with  an  eye  to  the  impact  on  cost, 
schedule,  and  performance. 
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The  researcher’s  conclusion  is  that  this  reveals  the  well  defined  and  rigid  nature 
of  the  DAS.  The  risks  inherent  to  the  DAS  are  well  known  and  understood  and  a  PM 
either  is  at  the  mercy  of  risks  beyond  his  control  or  can  expect  to  encounter  more  or  less 
the  same  type  of  risks  no  matter  the  type  of  program. 

3.  Definition  of  Uncertainty  in  the  Defense  Acquisition  System 

This  research  question  into  the  definition  of  uncertainty  originated  in  the  study  of 
the  hedging  of  risk  by  using  portfolio  management  techniques.  If  risk  can  be  quantified 
by  measuring  probability  of  occurrence  and  the  severity  of  the  event  happening,  (such  as 
the  Risk  Management  Guide  "s  Risk  Reporting  Matrix),  then  the  possibility  arises  that 
future  research  could  reveal  a  way  to  quantify  uncertainty  in  a  way  useful  for  PMs.  The 
means  of  measuring  things  of  which  we  have  incomplete  knowledge  is  beyond  the  scope 
of  this  thesis,  but  a  thorough  understanding  of  uncertainty  as  it  pertains  to  the  DAS  might 
give  insight  into  the  creation  of  an  useful  tool  or  method  of  addressing  uncertainty  such 
as  the  Chicago  Board  Options  Exchange  Market  Volatility  Index,  or  VIX,  which 
measures  the  thirty  day  expected  volatility  of  the  S&P  500.  Some  investors  follow  the 
VIX  in  order  to  gauge  the  general  investor  sentiment  that  may  imply  future  up  and  down 
swings  in  the  market,  with  greater  volatility  revealing  investor  uncertainty  and  lack  of 
confidence  (Chicago  Board  Options  Exchange,  2009).  VIX  uses  historical  price  data 
going  back  over  twenty  years  and  tracks  the  prices  of  put  and  call  stock  options.  If  the 
price  of  put  and  call  options  traded  on  the  market  can  be  used  to  gauge  investor  sentiment 
and  the  implied  short  term  course  of  the  market  up  or  down  there  might  be  a  facet  of  the 
DAS  that  could  be  utilized  in  a  similar  way. 

Beyond  the  dictionary  given  definition  there  is  not  much  thought  given  in  the 
DAS  to  the  definition  of  uncertainty  and  no  unique  interpretation  of  uncertainty  was 
revealed  in  the  interview  process.  When  the  researcher  bought  up  the  idea  of  a  future 
measurement  of  uncertainty  applicable  to  the  DAS  the  subject  only  elicited  minimal 
interest  and  the  subject  of  the  conversation  soon  changed  of  its  own  accord.  One 
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interview  subject  did  state  that  complexity  was  a  proxy  for  uncertainty  and  that  the 
Program  Management  technique  of  Progressive  Elaboration  was  a  means  of  dealing  with 
uncertainty  in  a  program. 

The  researcher’s  conclusion  is  that  beyond  the  dictionary  definition  of  uncertainty 
most  PMs  think  only  in  terms  of  risk  and  believe  that  any  measurement  of  uncertainty  in 
the  DAS  is  a  long  way  off.  PMs  are  well  infonned  on  Progressive  Elaboration  techniques 
and  the  need  to  invest  as  much  time  as  possible  up  front  in  a  program  in  order  to  reduce 
risks  and  uncertainty.  The  DAS  already  practices  a  basic  form  of  risk  mitigation  in  the 
face  of  uncertainty  by  preferring  fixed  price  contracts  when  a  program  has  lower  levels  of 
uncertainty  and  cost  type  contracts  for  programs  with  high  levels  of  uncertainty.  This  is 
an  area  in  which  more  study  and  understanding  is  required  before  any  useful 
measurement  or  tool  can  be  developed. 

4.  Questions  on  the  Impact  of  an  OA  Approach 

During  the  course  of  the  interview  process,  there  was  a  general  avoidance  of 
giving  opinion  concerning  OA  and  its  impact  on  the  DAS.  Even  when  pressed  with  the 
specific  questions  listed  in  the  Research  Questions  section  of  this  thesis  half  of  the 
interview  subjects  would  talk  in  generalities  claiming  that  they  had  not  much  experience 
with  OA  during  their  career.  For  those  interview  subjects  that  had  extensive  experience 
with  OA,  especially  those  working  at  PEO-IWS,  the  consensus  was  that  it  is  still  too 
early  to  tell  the  impact  of  OA  on  the  system  as  a  whole,  especially  if  OA  and  SOA  has 
produced  the  desired  results  and  efficiencies  in  the  DAS. 

The  researcher  also  came  across  a  misunderstanding  on  OA  and  at  what  level  in  a 
program  that  OA  resided.  During  one  interview,  when  asked  to  provide  an  example  of  a 
successful  use  of  OA  the  PM  instead  gave  a  casebook  example  of  an  open  business 
strategy.  His  program  acquired  extra  funds  by  utilizing  a  small  business  even  though 
their  hardware  was  not  initially  compatible  to  the  planned  architecture  of  the  system  and 
he  was  able  to  dividing  the  rest  of  the  project’s  hardware  requirements  and  the  update  of 
legacy  software  programs  between  two  large  defense  contractors.  The  interview  subject 
gave  a  good  -strategic”  overview  of  his  program  but  revealed  little  of  any  -tactical” 


44 


success  using  OA  principles  even  though  the  program  in  question  did  in  fact  have  to  use  a 
modular  approach,  allowing  integration  of  independently  developed  legacy  systems  with 
reliance  on  reuse  of  many  legacy  system  software  and  hardware  components. 

One  area  of  agreement  during  the  interview  process  was  that  OA  increased 
complexity  when  ensuring  that  systems  met  infonnation  security  and  information 
assurance  requirements,  though  a  lot  of  the  complexity  was  also  attributed  to  the  system 
and  network  accreditation  process.  Success  stories  such  as  the  Rapid  COTS  insertion 
process  were  mentioned  along  with  problems  in  implementing  a  support  system  for  OA 
centric  programs,  such  as  PEO-IWS’s  Share  portal,  but  the  general  reluctance  to 
pronounce  on  OA’s  effects  on  the  DAS  as  a  whole  remained.  The  researcher  interpreted 
the  general  inconclusiveness  of  the  findings  concerning  OA  in  the  DAS  that  the 
implementation  of  OA  was  still  in  its  initial  stages  and  that  the  supporting  tools,  systems, 
and  procedures  were  at  an  immature  level.  The  researcher  was  unsure  if  the  data 
collected  so  far  pointed  to  a  general  failure  of  OA  in  the  DAS  or  that  OA  would  be 
beneficial  for  certain  types  of  programs  and  unsuitable  for  others.  The  researcher  decided 
to  begin  secondary  research  into  historical  examples  of  OA  in  the  DAS  in  an  attempt  to 
find  the  answers  missing  during  the  active  interview  process. 
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IV.  OA  AND  RISK  AND  UNCERTAINTY  IN  THE  DOD 
ACQUISITION  SYSTEM 

This  chapter  will  explore  three  different  examples  of  the  implementation  of  OA 
and  SOA  principles  in  the  DoD  and  beings  discussing  the  Acoustic  Rapid  COTS 
Insertion  (A-RCI)  /  Advanced  Processor  Build  (APB).  A-RCI  began  an  attempt  to 
improve  the  detection  capability  of  towed -array  sonar  on  the  Los  Angeles  class  of  attack 
submarines  and  soon  expanded  to  include  all  sonar  systems  in  the  submarine  fleet  along 
with  some  surface  and  aviation  submarine  detection  systems.  Exploring  the  A-RCI 
program  is  worthwhile  as  it  provides  a  good  example  of  the  successful  implementation  of 
MOSA  and  OA  principles  and  is  considered  the  -poster  child”  for  OA  in  the  Navy 
(CHIPS,  2007).  A-RCI  is  an  example  of  the  use  of  OA  and  SOA  to  improve  an 
operational  capability  of  the  anned  forces  in  a  rapid  manner,  from  awarding  of  the  initial 
contracts  to  the  delivery  of  new  capabilities  to  the  fleet.  In  the  following  section,  an 
example  is  given  of  how  a  SOA  strategy  has  solved  daunting  integration  issues.  The  U.S. 
Military  Entrance  Command  (USMEPCOM)  is  a  joint  command,  consisting  of  elements 
from  all  the  Armed  Services,  to  include  their  reserve  components  as  well  as  the  U.S. 
Coast  Guard  that  relies  on  each  service’s  legacy  recruiting  system  to  process  their 
applicants.  A  SOA  strategy  was  used  to  allow  the  smooth  flow  of  data  from  legacy 
recruiting  systems  into  USMEPCOM’s  system. 

The  chapter  will  finish  with  a  discussion  of  PEO-IWS’s  Software  Hardware  Asset 
Reuse  Enterprise  (SHARE)  repository  which  is  an  online  portal  containing  a  library  of 
combat  system  software  and  related  assets  for  use  by  contractors  for  developing  or 
suggesting  improvements  to  Navy  Surface  Warfare  Systems  (Blais,  2008).  Not  only  is 
the  SHARE  initiative  directly  supporting  the  SOA  and  OA  principles  of  reusability  but  it 
is  part  of  a  larger  overall  attempt  to  build  a  new  infrastructure  within  the  DAS,  an 
infrastructure  consisting  of  new  tools,  web  portals,  and  ways  of  doing  business  that  will 
allow  future  projects  and  programs  to  use  OA  with  the  minimum  of  friction. 
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A.  ACOUSTIC  RAPID  COTS  INSERTION 

The  Russian  Akula  class  submarine  is  making  10  knots  on  course  275  under  the 
Eastern  Mediterranean  Sea  about  200  miles  off  the  Syrian  Coast.  Captain  Spravtsev, 
commanding  officer  of  the  Volk  is  nervously  waiting  for  reports  on  a  suspected  contact, 
hopefully  one  of  the  American  Los  Angeles  class  submarines.  On  patrol  in  support  of  the 
Admiral  Kuznetsov  battle  group,  the  Volk  has  been  attempting  to  track  the  NATO 
submarines  that  are  themselves  conducting  reconnaissance  on  the  Russian  aircraft 
carrier’s  activities. 

Earlier  in  the  evening,  the  Officer  of  the  Deck  had  called  him  to  the  Control 
Room.  Upon  entering  the  cramped  space  and  noticing  the  smiling  faces  and  hushed 
commotion  around  the  sonar  operator  Spravtsev  only  needed  a  second  to  guess  that  they 
had  a  significant  contact.  The  CO  had  to  gently  push  the  Officer  of  the  Deck  and  an  off 
duty  sonar  operator  aside  in  order  to  reach  the  console.  In  an  excited  whisper,  the  sonar 
operator  told  him  that  they  were  now  tracking  a  Los  Angeles  Class  submarine.  The  Volk 
had  managed  to  track  the  688  Class  sub  for  over  two  hours  but  now  had  lost  contact.  It 
had  been  over  thirty  minutes,  and  Spravtsev  was  nervous  that  the  American  submarine 
had  suspected  they  were  being  tracked  or  had  acquired  his  own  sub  as  a  contact. 
-Conning  Officer  slow  to  five  knots.”  Spravtsev  knew  that  his  sub  had  an  improved  hull 
and  was  quieter  than  the  first  seven  submarines  in  the  class  and  that  if  he  slowed  down 
the  Americans  might  not  suspect  he  was  there. 

The  American  submarine  had,  in  fact,  acquired  the  Volk’s  acoustic  signature  and 
had  been  tracking  Spravtsev’s  command  for  over  an  hour.  There  was  the  same  level  of 
hushed  excitement  of  picking  up  one  of  the  new  Akulas  as  a  contact.  Soon  after 
Spravtsev  gave  the  order  to  slow  his  submarine,  the  excitement  onboard  the  American 
submarine  died  away  and  the  watch  team  got  down  to  the  serious  work  of  trying  to 
reacquire  the  contact.  Unlike  their  experience  with  the  Victor  Class  submarines,  the 
Volk's  improved  acoustic  profile  prevented  the  American  submarine  from  acquiring  her 
contact  until  after  she  surfaced  two  days  later  (Cole,  2011). 
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1.  The  USN’s  Response  to  the  Loss  of  Acoustic  Superiority 

In  1995,  while  testifying  before  the  House  National  Security  Committee  on  the 
Seawolf  submarine  program  for  the  1996  defense  budget,  then  Chief  Of  Naval 
Operations,  Admiral  Jeremy  Boorda,  stated  that  the  Russian  Navy  was  operating  six 
submarines  that  were  quieter  than  the  U.S.  Navy’s  then  state-of-the-art  class  of 
submarines,  the  Los  Angeles  class.  He  described  the  difficulty  that  U.S.  submarine 
commanders  had  in  tracking  the  newer  Russian  Akula  class  submarines  at  slower  speeds 
and  expressed  concerned  over  the  proliferation  of  the  quiet  Kilo  class  electric  diesel 
submarines  in  hostile  countries’  submarine  forces.  The  fictional  account  of  the  American 
submarine  losing  its  target  after  Captain  Spravtsev  slowed  his  submarine  had  occurred  all 
too  often  in  real  life  operations  and  was  unnerving  for  a  service  that  had  relied  on 
technical  superiority  to  make  up  for  a  numerical  inferiority. 

Admiral  Boorda  was  not  only  making  the  best  case  for  spending  limited  defense 
dollars  for  the  Seawolf  submarine  but  articulating  the  concern  over  the  recent  loss  of  the 
traditional  U.S.  acoustic  superiority  and  submarine  quieting  technology  over  the  world’s 
navies.  The  loss  of  this  vital  edge  in  submarine  warfare  came  at  an  inopportune  time  as 
the  Navy  was  also  fighting  increased  costs  in  developing  new  weapons  systems,  reduced 
budgets  as  part  of  the  drawdown  after  the  Cold  War,  debate  over  the  future  of  the 
Seawolf  submarine,  and  subsequent  decision  to  cancel  the  Seawolf  submarine  program  in 
favor  of  the  Virginia  Class. 

This  sense  of  crisis  over  the  lost  technological  edge  and  the  need  to  regain  it  in  a 
time  of  financial  constraints  opened  the  door  for  unique  and  creative  approaches  to 
solving  the  problem  that  went  against  the  structure  and  intent  of  many  of  the  DAS  and 
JCIDS  requirements.  In  1996,  the  PEO  for  submarines  and  the  PM  for  the  new  Virginia 
class  C3I  (now  C4I)  conceived  of  the  following  guiding  principles  for  a  new  initiative 
that  would  turn  into  the  A-RCI  and  APB  projects  (Udicious,  2004): 

•  Rapid  COTS  insertion  means  just  that. 

•  Deliver  each  sensor’s  full  theoretical  gain  to  the  operator  -all  bearings,  all 
frequencies,  all  the  time. 
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•  Avoid  modifying  successful  commercial  products. 

•  Use  the  lessons  learned. 

•  Use  state  of  the  practice,  not  state  of  the  art  systems;  tactical  sonar 
systems  are  not  beta  test  sites. 

•  Configuration  management,  not  configuration  control. 

•  Software  reuse  is  the  key  to  affordability. 

•  No  single  organization  has  the  full  story. 

•  Sub  acoustic  superiority  depends  on  the  successful  use  of  these  axioms. 

Many  of  the  founding  principles  of  A-RCI  are  directly  related  to  the  principles 
and  essential  performance  characteristics  of  OA,  SOA,  and  MOSA.  Software  reuse 
needs  no  explanation  but  collaboration  is  mandated  by  the  warning  that  no  organization 
-has  the  full  story”  and  the  use  of  state  of  the  practice  vice  state  of  the  art  is  key  in 
ensuring  interoperability,  especially  in  a  project  operating  on  a  compressed  schedule. 
Using  the  lessons  learned  by  other  organizations,  to  include  the  lessons  learned  in  the 
civilian  world  helped  to  achieve  a  consensus  based  approach  instead  of  the  DoD  trying  to 
solve  the  problem  with  a  proprietary  solution. 

2.  A-RCI/ APB  Development  Strategy 

The  existing  sonar  systems  at  the  time  were  not  designed  with  modularity  in  mind 
and  one  of  the  first  A-RCI  tasks  was  to  update  the  legacy  systems.  This  introduced  some 
short  term  operational  risk  to  the  submarine  forces  as  the  first  APB  only  allowed  the 
towed  sonar  array  to  be  operated  at  a  single  display  station  rather  than  one  many  of  the 
other  displays  available  (Boudreau,  2006).  As  part  of  the  MOSA  strategy,  system 
improvements  were  divided  between  the  hardware  and  software  components  allowing  for 
the  use  of  COTS  for  hardware  upgrades  and  the  application  software  developed 
independently  from  the  processors  by  the  use  of  transportable  middleware.  This  allowed 
for  quicker  development  and  fielding  of  new  upgrades  and  gave  the  developers  the 
flexibility  to  change  some  parts  of  the  system  while  leaving  others  alone  (Boudreau, 
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2006).  There  were  risks  involved  with  the  MOSA  approach  as  system  development  for 
different  components  proceeded  independently  of  each  other  and  introduced 
interoperability  risks  into  the  process.  Extra  time  and  expense  was  needed  on  tracking 
and  version  control  of  key  software  interfaces,  standards,  and  protocols  amongst  the 
different  development  teams. 

A-RCI  used  an  open  capabilities  based  business  model  as  opposed  to  the 
requirements  based  business  model  that  informs  much  of  JCIDS  (Udicious,  2004).  In 
short,  A-RCI  would  seek  to  use  the  technology  available  at  the  time  instead  of  trying  to 
either  develop  new  technologies  or  improve  a  system  in  order  to  meet  specific  user 
requirements.  A  spiral  development  model  was  chosen  that  included  a  build/test/build 
sequence  with  any  new  system  additions  required  to  pass  through  a  thorough 
demonstration  process  that  included  the  evalution  of  new  system  capabilites  against  the 
previous  system  capabilities  though  collaboration  and  information  sharing  were  expected 
(Boudreau,  2006). 

Software  development  operated  on  an  annual  upgrade  cycle  through  the  APB  and 
the  hardware  would  be  selected  operated  on  a  biannual  schedule  which  was  described  in  a 
study  as  a  -highly  demanding  acquisition  op  tempo”  (Boudreau,  2006).  This  high  op 
tempo  was  in  conflict  with  the  need-driven  JCIDS  and  event  driven  DAS  though  the 
sense  of  crisis  of  the  loss  of  acoustic  superiority  at  sea  had  the  benefit  of  giving  A-RCI 
high  level  support  allowing  for  the  bypassing  of  various  milestones  and  requirements  in 
JCIDS  and  the  DAS  in  favor  of  PM  flexibility,  technical  innovation,  and 
experimentation.  The  priority  was  a  quick  development  cycle  and  release  of  improved 
sonar  capabilities  to  the  operating  forces  and  if  contractors  could  not  stay  on  schedule, 
they  were  left  behind  for  that  development  spiral  (Boudreau,  2006). 

A-RCI  followed  an  innovative  approach  in  order  to  leverage  the  benefits  of 

collaboration  between  contractors,  both  small  and  large,  academic  laboratories  and 

government  organizations.  Lockheed  Martin  served  as  the  prime  contractor  for  A-RCI 

but  the  focus  was  changed  to  be  a  -prime  system  integrator.”  Even  though  Lockheed 

Martin  would  play  the  major  role  in  the  contract,  the  door  was  opened  to  smaller 

contractors  and  other  organizations  that  usually  could  not  or  would  not  participate  in  the 
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acquisition  process.  The  main  vector  for  input  from  small  contractors  and  nontraditional 
entities  into  A-RCI  was  the  peer  review  process  that  selected  between  different 
alternatives  and  chose  the  best  solution,  usually  after  testing  with  real  world  data 
(Boudreau,  2006).  This  strategy  arose  out  of  one  of  the  founding  guiding  principles  of 
the  program  in  that  -no  single  organization  has  the  whole  story.”  The  peer  review 
process  was  conducted  under  the  over  site  of  a  Navy  PM  with  the  goal  of  preventing  the 
usual  tendency  for  the  prime  contractor  to  mold  the  program  in  the  most  profitable 
direction  for  it  instead  of  ignoring  competitor’s  solutions  that  may  have  been  more 
suitable.  The  peer  group  structures  were  designed  for  flexibility  and  an  extensive  set  of 
working  groups  were  set  up  to  cover  most  aspects  of  the  program  to  include  a  Tactical 
Integration  Advisory  Group,  groups  for  specific  sub  systems  such  as  the  APB-1/2  towed 
array,  and  perhaps  most  importantly  an  Operator  Feedback  group.  The  composition  of 
the  groups  was  fluid  over  the  project  life  cycle  with  groups  merging  or  even  disbanding 
depending  on  the  circumstances  (Boudreau,  2006). 
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Figure  5.  A-RCI  System  Development  Model  (From  Barron,  2006) 

Figure  5  presents  an  overview  of  the  A-RCI  development  model  with  the  USN 


providing  the  operational  requirements  and  needed  performance  specifications  to  the 
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collaborative  heart  of  the  program  with  the  APB  program  depending  on  competitive 
selection  and  test  on  an  annual  cycle,  adjacent  to  but  not  fully  dependent  on  the  six  month 
COTS  hardware  development  cycle.  The  spiral  development  method  hoped  to  leverage 
the  increases  in  technology  by  using  cheaper  COTS  components  on  a  rapid  upgrade  cycle 
in  order  to  keep  up  with  the  comparatively  rapid  technological  advances  in  civilian 
industry.  The  spiral  process  would  continue  until  the  USN  regained  acoustic  superiority 
at  sea. 


3.  A-RCI  and  Associated  Risks 

The  risks  associated  with  A-RCI  were  generated  by  the  unique  development 
strategies  pursed  by  the  PMs  and  the  friction  with  the  traditional  developmental  approach 
following  JCIDS  and  DAS  best  practices.  These  risks  can  be  assumed  relevant  to  any 
other  program  following  using  MOSA  and  OA  principles  and  using  a  spiral  development 
strategy. 


a.  Budget  Risks 

The  initial  A-RCI  program  was  funded  at  level  less  than  equivalent 
programs  using  a  traditional  approach  and  had  different  funding  profiles  necessitating 
-eontinuous  streams”  of  RDT&E,  Procurement,  and  Operations  and  Support  accounts 
(Boudreau,  2006).  The  mitigation  to  this  budget  risk  was  quick  initial  success  and 
delivery  of  improved  sonar  capability  to  the  fleet  sooner  than  if  the  program  followed  the 
usual  path.  The  initial  delivery  of  the  system  to  the  operational  forces  was  not  the  final 
solution  to  the  problem  but  it  did  justify  the  funds  allocated  for  the  program  and  made  it 
easier  for  the  PMs  to  acquire  more. 

b.  Program  Risks 

The  emphasis  given  to  schedule  over  perfonnance  generated  many  types 
of  program  risk.  The  crisis  driven  focus  on  rapid  deployment  of  new  sonar  technologies 
and  reliance  on  future,  though  unproven  at  the  time,  cost  benefits  using  a  COTS 
development  strategy  was  bound  to  cause  friction  between  the  program  and  elements  of 
the  DAS  and  JCIDS  in  place  to  reduce  risk  exposure  to  performance  goals. 
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Even  though  A-RCI’s  Peer  Review  Groups  were  an  integral  part  of  a 
successful  program,  there  was  always  risk  of  stalemate  within  or  between  the  various 
groups.  The  best  mitigation  factor  against  this  risk  was  that  all  groups  operated  under  the 
guidance  of  a  Navy  PM,  who  could  cast  a  deciding  vote  and  take  ultimate  responsibility 
for  a  final  decision. 

Testing  played  a  vital  role  in  A-RCI,  especially  in  the  build/test/build 
development  phases  and  operational  testing  at  sea  though  end  to  end  testing,  so  important 
in  the  DAS  to  verifying  performance  goals  turned  out  to  increase  cost  and  schedule  risks 
in  light  of  the  higher  op  tempo.  This  testing  is  especially  problematic  in  a  spiral 
development  program  that  will  proceed  without  any  unready  component  at  the  end  of  a 
particular  phase  (Boudreau,  2006). 

c.  Integration  Risks 

The  rapid  op  tempo  of  A-RCI  was  incongruent  with  the  slower  JCIDS 
cycle.  JCIDS  helps  ensure  that  a  program  addresses  mission  needs  that  originated  in  the 
NSS  and  prevents  costly  redundancy  in  systems  procurement,  especially  redundant 
systems  procured  by  different  services.  A-RCI’s  rapid  pace  resulted  in  incomplete 
reviews  threatening  future  integration  of  the  weapons  system  beyond  the  submarine  force 
and  potential  to  operate  in  a  joint  environment.  The  mitigation  was  to  conduct  annual 
JCIDS  reviews  synchronized  to  sequential  development  spirals  (Boudreau,  2006).  This 
was  possible  because  subsurface  operations  and  anti-submarine  warfare,  besides  strategic 
nuclear  forces,  is  almost  wholly  an  USN  mission.  The  question  remains  open  if  such  an 
accommodation  would  be  possible  with  JCIDS  if  the  program  in  question  would  rapidly 
deploy  an  anti-aircraft  or  missile  system  that  would  be  certain  to  operate  in  a  joint 
environment. 


d.  Operational  Risks 

The  first  operational  risk  was  that  the  initial  deployment  of  the  system 

would  not  meet  the  operational  requirements  of  the  fleet.  The  sonar  system  deployed 

after  the  first  development  cycle  could  only  be  accessed  from  a  single  console,  which 

hampered  the  ability  of  the  watch  team  to  access  infonnation  in  a  timely  manner.  The 
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operators  in  the  fleet  were  able  to  deal  with  the  inconveniences  and  risks  until  subsequent 
releases  were  deployed  with  better  accessibility. 

Operator  and  maintenance  technicians  also  had  to  spend  extra  time  and 
effort  in  learning  about  each  new  iteration  of  the  sonar  system.  When  the  cost  and  time 
burden  is  placed  upon  the  operational  user  it  does  not  appear  in  the  cost  and  schedule 
metrics  of  the  systems  program.  A-RCI’s  success  can  be  attributed  to  an  emphasis  on 
operator  feedback  with  working  groups  devoted  to  their  concerns.  The  risk  for  future 
programs  using  a  spiral  approach  is  that  cost  and  schedule  burdens  may  actually  be 
transferred  to  the  operating  forces  where  feedback  might  not  flow  back  in  time  to  allow 
adjustments  in  the  developmental  cycle. 

e.  The  A-RCI  Success  Story 

Throughout  the  USN  A-RCI  is  considered  a  MOSA  /  OA  success  story. 
In  2006,  then  CNO,  Admiral  Mullen  sent  a  memo  to  the  heads  of  various  system 
commands  and  urged  then  to  use  follow  the  best  practices  of  the  A-RCI  program.  He 
stressed  the  importance  of  going  beyond  using  an  OA  approach  to  technical  problems  and 
use  an  open  business  approach  for  -the  acquisition  and  spiral  development  of  new 
systems  that  enable  multiple  developers  to  collectively  and  competitively  participate  in 
cost  effective  and  innovative  capability  to  the  Naval  enterprise”  (Mullen,  2006). 

Some  of  the  measurable  effects  of  A-RCI  included  the  following 
improvements  to  the  USN’s  operating  forces  aucoustic  capabilites  (Boudreau,  2006): 

•  Sevenfold  increase  in  processing  capability. 

•  Mean  operator  success  rate  increased  by  a  factor  of  four. 

•  Mean  number  of  false  alanns  reduced  by  40%. 

•  Detection  and  classification  time  improved  by  27  minutes. 

•  Mean  hold  time  improved  by  25  minutes. 
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Life-cycle  costs  were  improved  by  a  factor  of  close  to  five  to  one 
(Boudreau,  2006)  with  many  of  the  savings  attributed  to  the  implementation  of  the  COTS 
strategy  (Udicious,  2004).  Risks  taken  to  allow  for  the  quick  delivery  of  operational 
improvements  to  the  fleet  paid  off. 

Integration  risks  were  taken  but  they  were  always  minimal  as  A-RCI  was 
developing  the  next  generation  of  sonar  systems  and  future  surface  and  aviation  sonar 
detection  systems  would  be  integrating  with  A-RCI,  not  the  other  way  around. 
Leadership  was  confronted  with  the  need  to  maintain  a  delicate  balance  between 
innovation  stifling  centralization  and  bureaucracy  or  the  anarchy  of  too  few  interface 
definitions  that  would  doom  the  integration  of  the  different  developments.  A-RCI’ s  OA 
approach  called  for  -parallel  developments  by  different  organizations,  using  independent 
software  development  tools,  and  funded  by  multiple  contracts”  An  integrated  product 
team  approach  was  used  that  used  collaboration  between  the  government,  industry  and 
even  academic  labs  to  keep  an  element  of  coordination  between  the  different 
organizations  (Udicious,  2004). 

Program  risk,  especially  the  risk  inherent  in  a  collaborative  system  were 
overcome  through  strong  leadership,  especially  senior  leadership  providing  -top  cover” 
in  allowing  middle  management  to  innovate  (Boudreau,  2006)  and  intervening  to  break 
any  stalemate  in  the  various  peer  review  groups.  Input  from  the  fleet  was  vital  to 
providing  program  leadership  and  the  peer  review  groups  on  what  was  working  and  what 
wasn’t  working  so  any  arguments  within  the  various  peer  review  and  other  development 
groups  were  less  theoretical  and  more  technically  based  on  the  operator’s  needs.  The 
spiral  development  cycle  introduced  cost  and  schedule  risks  but  since  A-RCI  was  a 
software  and  computer  processor  intensive  program  the  spiral  development  strategy  fitted 
in  nicely  with  the  rapid  increases  in  processing  power  and  technical  advances  that  were 
widespread  in  the  civilian  academic  and  business  realms. 

Budget  and  cost  risks  are  inherent  in  any  spiral  development  strategy.  In 

A-RCI’s  case,  success  lead  to  success  and  as  the  initial  deployments  of  new  sonar 

technology  showed  rapid  improvement  in  the  operational  forces’  acoustic  capabilities  as 

the  program  matured  it  became  more  and  more  likely  that  the  spiral  development  model 
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would  not  bog  down  into  repetitive  cycles  of  minimal  improvements  with  hardly 
noticeable  technological  improvements.  During  the  interview  process,  the  researcher 
uncovered  a  common  belief  amongst  program  managers  that  the  longer  a  system’s 
development  cycle,  more  risk  is  added  and  there  is  a  higher  overall  chance  of  program 
failure.  The  spiral  nature  of  A-RCI  with  a  series  of  short  development  cycles,  delivered 
to  the  fleet  the  technological  improvements  available  at  that  time  as  opposed  to  waiting 
for  years  to  deliver  a  supposedly  state  of  the  art  system.  One  key  to  the  success  of  A- 
RCI’s  spiral  strategy  can  be  summed  up  to  a  PM’s  comment  during  the  interview  process 
that  -sometimes  good-enough  is  good-enough.”  Instead  of  investing  time  and  money 
into  improving  a  system  (in  the  software  engineering  field  it  is  well  known  that  most 
costs  of  software  development  are  incurred  after  most  of  the  programming  has  been 
done). 

B.  U.S.  MILITARY  ENTRANCE  COMMAND  INTEGRATION  PROBLEM 

WITH  SOA  SOLUTION 

USMEPCOM’s  mission  is  to  review  all  applications  of  recruits  to  the  U.S.  Armed 
Forces  and  then  process  their  records  from  the  initial  interview  with  a  recruiter  until  the 
new  accession  reports  to  their  training  facility.  USMEPCOM  processes  approximately  1 
million  records  a  year  and  at  any  given  time  stores  over  60  million  records  across  all  the 
armed  services.  Over  15,000  recruiters  and  3,000  GS  employees  use  the  system  to 
process  the  applications  and  exchange  data  with  outside  federal  agencies,  such  as  the  FBI 
and  even  the  Department  of  Motor  Vehicles  from  all  50  states  (Maravola,  2009). 
Internally,  the  USMEPCOM  network  serves  65  Military  Entrance  Processing  Stations 
and  500  Military  Entrance  Testing  sites  (U.S.  Military  Entrance  Processing  Command 
[USMEPCOM],  2007) 

Each  application  is  processed  by  a  recruiter  from  the  particular  branch  of  the 
Armed  Services  the  recruit  is  joining  and  each  branch  uses  their  own  propriety  system. 
Extra  time  and  effort  was  spent  by  the  recruiters  in  the  redundant  entry  of  data  into 
USMEPCOM’s  system,  which  could  only  be  accomplished  by  manually  entering  the  data 
into  a  flat  file  for  upload  into  USMEPCOM’s  database  or  a  tedious  double  entry  directly 
into  USMEPCOM’s  system  soon  after  entering  the  system  into  the  recruiter’s  system. 
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1. 


DE/TO  SIP  Overview 


USMEPCOM’s  response  to  the  integration  problem  between  its  system  and  the 
various  armed  services’  recruiting  networks  was  called  the  Data  Exchange/Top  of  System 
Interface  Process  (DE/TOSIP).  The  initial  goal  of  the  program  was  the  reduction  of 
reducing  application  processing  time  from  2.6  days  to  less  than  a  day,  and  allowing  the 
prequalification  of  90%  of  the  applicants  without  the  applicant  having  to  visit  a  central 
facility  in  person  such  as  a  Military  Entrance  Processing  Station  (MEPS)  This  would  save 
a  significant  amount  of  time  and  expense;  especially  in  recruiting  regions  where  the 
MEPS  facility  may  be  over  100  miles  journey  from  the  recruiting  office.  The  chosen 
strategy  was  to  use  an  -accepted  standards  based  SOA  interface”  that  was  compliant  with 
all  Defense  Information  System  Agency  (DISA)  system  requirements  (Maravola,  2009). 

The  DE/TOSIP  team  contained  representatives  from  USMEPCOM 
administration.  Information  Technology,  budgeting,  and  contractor  engineers  and  soon 
grew  to  include  representatives  from  the  military  services  and  Federal  agencies.  Their 
development  plan  was  based  on  using  well  known  commercial  software  from  Oracle  as 
the  interface  solution  between  USMEPCOM  and  the  recruiting  systems  of  the  various 
branches  of  the  armed  services.  Though  the  literature  does  not  explicitly  state  it,  it  is  the 
researcher’s  conclusion  that  Oracle  software  was  chosen  by  the  team  for  a  variety  of 
reasons  to  include  cost  savings  using  a  COTS  solution  from  a  well  known  and  popular 
vendor,  the  frequency  in  which  Oracle  products  were  used  in  the  myriad  of  systems  that 
DE/TOSIP  would  be  required  to  exchange  data  with,  and  with  an  eye  on  future  reuse  of 
system  components  and  ease  of  system  scalability.  This  conclusion  was  confirmed  in  an 
interview  with  an  USMEPCOM  acquisition  professional  in  the  Acquisition  and  Initiatives 
Division  serving  at  the  time. 

During  the  interview  another  important  strategy  came  to  light  in  that  the  program 
was  purposefully  split  into  three  smaller  programs  to  avoid  the  budget  threshold  requiring 
a  PM,  and  compliance  with  DAS  milestones  that  would  slow  the  project  down.  The  first 
step,  begun  in  2006,  was  to  implement  SOA  principles  in  USMEPCOM’s  network  and 
demonstrate  the  practicability  of  the  new  system  by  generating  service  calls  internally.  In 

2007,  the  second  -program”  was  the  electronic  conversion  of  all  paper  records  in 
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preparation  for  direct  access  from  a  recruiter’s  workstation  to  the  information,  bypassing 
the  need  for  a  representative  at  USMEPCOM’s  facilities  to  answer  the  recruiter’s  data 
request.  In  2008,  the  final  program  was  the  implementation  of  electronic  security  and 
privacy  data  features  for  the  processing  of  recruits  personal  information  and  the  ability  to 
gather  and  process  biometric  data.  In  comparison,  the  program  passed  the  budget 
threshold  in  2008  and  acquired  a  PM.  The  development  cycle  has  since  slowed  due  to 
the  many  bureaucratic  constraints  as  discussed  earlier  in  this  thesis. 

During  the  interview  with  the  USMEPCOM  acquisition  professional  the  topic  of 
risks  that  the  DE/TOSIP  IPT  encountered  was  discussed.  The  main  risks  to  the  program 
were  scope  creep  on  the  part  of  both  the  end  users  and  USMEPCOM  and  getting  the  end 
users  to  buy  into  the  proposed  system  solutions.  The  manager  described  how  the  general 
officers  representing  the  different  service  branches  were  usually  happy  with  the  proposed 
solutions  but  it  was  problematic  to  get  buy  in  from  those  that  had  to  use  the  system  on  a 
day  to  day  basis.  The  mitigation  strategy  against  scope  creep  was  to  define  the 
requirements  with  all  parties  at  the  beginning  of  the  process  and  hold  to  those 
requirements  unless  the  party  requesting  new  features  would  provide  funding.  Gaining 
user  buy  in  was  a  longer  process  and  required  tact  and  diplomacy  by  the  IPT  and 
necessitated  a  -Tot  of  talking.”  There  were  no  shortcuts  to  gaining  user  trust  and 
confidence  available  and  the  time  and  effort  spent  in  engaging  the  true  end  users  paid  off. 

2.  DE/TOSIP  Results 

The  U.S.  Air  Force  Reserve  estimated  that  they  save  around  $350  annually  from 
their  recruiting  budget  as  a  recruiter  can  now  retrieve  applicant  data  online  instead  of 
having  to  call  the  Air  Force  liaison  at  a  USMEPCOM  facility.  Since  the  Air  Force 
Reserve  is  only  a  small  component  of  the  total  DoD  recruiting  force  (about  2  percent)  the 
estimated  savings  for  the  other  service  components  is  substantial  (Maravola,  2009). 

Any  system  can  benefit  from  significant  cost  savings  when  it  is  automated  but 
DE/TOSIP ’s  ability  to  quickly  access  data  from  multiple  systems  using  Oracle  software 
as  the  key  interface  between  systems  benefits  from  a  parallel  to  Metcalfe’s  Law  in  that 
the  usefulness  of  the  network  increases  by  each  outside  network  it  exchanges  data  with. 
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The  DE/TOSIP  development  team  benefited  from  software  reuse  when  designing 
a  security  module  and  in  building  the  interface  between  USMEPCOM’s  system  and  the 
different  Armed  Services’  Recruiting  systems.  A  third  party  analysis  of  the  project  found 
that  the  costs  of  enabling  a  virtual  integration  system  saved  about 
$56  million  due  to  the  SOA  (Maravola,  2009). 

C.  SHARE  AND  THE  DEVELOPMENT  OF  AN  INFRASTRUCTURE  TO 

SUPPORT  OA 

A-RCI  was  a  ground  breaking  program  for  OA  in  the  DAS  but  the  ability  of 
future  programs  to  benefit  from  the  principles  of  OA  will  be  helped  or  hindered  by  the 
supporting  infrastructure  in  the  acquisition  environment.  This  infrastructure  can  be 
anything  from  the  recommendation  of  a  JCIDS  supplement  concerning  rapid  op  tempo 
spiral  development  programs  (Boudreau,  2006),  collaboration  portals,  the  OA 
implementation  assessment  tool,  and  software  reuse  repositories. 

This  section  will  describe  the  Software  Hardware  Asset  Reuse  Enterprise 
(SHARE)  Repository  Framework  and  discuss  some  of  the  issues  encountered  in  the 
implementation  of  this  collaborative  initiative.  The  issues  in  establishing  infrastructure 
supporting  one  of  the  principles  of  OA  are  relevant  to  other  future  initiatives  in  building 
an  OA  friendly  infrastructure. 

1.  Overview  of  SHARE 

The  SHARE  repository  was  created  in  August  2006  under  the  auspices  of  PEO- 
IWS,  the  USN’s  lead  for  OA.  SHARE’S  goal  was  not  just  to  enable  the  reuse  of  combat 
system  software  but  also  related  assets  and  to  facilitate  prime  and  subcontractors  ability 
to  reuse  software  and  suggest  improvement  to  Navy  surface  warfare  systems  (Johnson  & 
Blais,  2008). 

The  SHARE  library  usues  an  online  open  source  repository  called  SourceForge 

that  can  also  be  used  as  a  project  management  tool.  The  library  contains  artifacts  from 

various  programs  to  include  the  AEGIS  ship  self  defense  system,  DDG-1000,  and  the 

Literal  Combat  Ship  program  and  is  divided  into  classified  and  unclassified  sections  . 

Library  materials  are  accesible  online  or  through  the  delivery  of  physical  media  (Johnson 
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&  Blais,  2008).  Before  a  contractor  or  government  agency  accesses  library  materials 
both  a  license  agreement  and  Non-disclosure  agreements  must  be  signed  and  saved  for 
future  reference. 

2.  Collaborative  Issues 

When  discussing  SHARE  during  interviews  with  PEO-IWS  personnel  it  was 
discovered  that  not  all  contractors  were  willing  to  post  artifacts  to  the  library,  as  they 
were  concerned  about  their  intellectual  and  proprietary  rights.  PEO-IWS  noticed  that  the 
traditional  larger  and  well  established  contractors  were  more  willing  to  participate  but 
smaller  firms  tended  to  avoid  posting  artifacts  to  the  library.  One  of  the  goals  of  SHARE 
was  to  attract  small,  innovative  firms  that  had  never  worked  with  the  DAS  allowing  the 
DoD  access  to  sources  of  new  technologies  and  system  development  strategies.  SHARE 
would  not  only  be  a  software  reuse  portal  but  a  marketplace  of  ideas  in  which  a  small 
company  could  post  an  artifact  and  essentially  -shop”  it  to  either  the  DoD  or  another 
contractor.  In  practice,  this  never  happened,  as  smaller  companies  were  afraid  of  losing 
any  technical  advantage  developed  internally  to  larger  companies.  Smaller  companies 
did  not  have  the  resources  to  fight  protracted  legal  battles  and  many  companies  relied  on 
a  very  limited  portfolio  for  their  financial  well  being.  There  were  many  benefits  to 
utilizing  SHARE  but  the  financial  and  intellectual  property  risks  were  too  great  for  many 
of  the  smaller  firms. 

PEO-IWS’  initial  response  to  this  problem  was  to  change  the  user  agreement 
forms  to  account  for  intellectual  property  rights  and  provide  more  oversight  on  the 
license  agreements  and  Non-disclosure  agreements  required  by  all  users. 

3.  Oversight  Issues 

The  attempt  to  solve  the  collaborative  issues  by  applying  more  oversight  over  the 

required  license  and  Non-disclosure  agreements  led  to  the  second  issue  to  hinder  the 

successful  implementation  of  SHARE.  PEO-IWS  personnel  discovered  that  a  significant 

percentage  of  contractors  were  not  keeping  records  of  the  various  agreements  and 

considerable  effort  had  to  be  applied  in  order  to  get  companies  to  comply.  Planning  for 

SHARE  did  not  account  for  a  much  increased  workload  or  for  new  positions  to  be  added 
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to  PEO-IWS  and  the  work  load  to  oversee  compliance  with  the  paperwork  requirements 
fell  on  a  workforce  that  was  already  fully  tasked. 

There  are  a  few  possible  solutions  to  this  problem,  but  all  solutions  have  serious 
drawbacks.  The  first  solution  would  be  to  penalize  companies  for  breaking  user 
agreements  and  not  complying  with  the  mandatory  requirements.  The  drawback  is  that 
this  could  draw  the  government  into  legal  issues  and  even  have  the  government  take  the 
side  of  one  contractor  over  another.  This  approach  would  tend  to  many  contractors  away 
from  participating  in  SHARE  and  would  hinder  its  development. 

A  second  solution  to  the  oversight  and  compliance  problem  would  be  to  fund 
extra  positions  at  PEO-IWS  dedicated  to  the  active  management  of  user  agreements  and 
the  protection  of  intellectual  property  rights.  In  effect,  PEO-IWS  would  be  a  neutral 
referee  and  keep  subtle  pressure,  as  opposed  to  applying  penalties,  on  contractors 
delinquent  in  their  paperwork  requirements.  This  approach  would  be  effective  but  PEO- 
IWS’  budget  does  not  allow  for  this  and  increases  in  funding  are  unlikely. 

The  solution  that  PEO-IWS  is  working  on  is  one  of  trying  to  incentive  compliance 
by  contractors  with  the  license  and  Non-disclosure  agreements  compliance.  As  of  the 
interview  process  for  this  thesis,  PEO-IWS  has  decided  that  it  will  see  this  approach  but 
is  still  in  the  planning  stage  on  the  best  approach  to  implement  this  solution. 

4.  Impact  of  SHARE 

Unlike  A-RCI,  SHARE  currently  cannot  be  considered  a  success.  During  the 
interview  process  with  PEO-IWS  personnel,  the  majority  of  the  conversation  revolved 
around  the  problems  that  they  faced  in  implementing  the  system.  In  the  literature  review 
for  this  thesis  the  researcher  encountered  very  few  mentions  of  SHARE  outside  of  PEO- 
IWS  authored  material. 

Despite  the  ongoing  problems  with  the  implementation  of  SHARE  and  the  lack  of 
well  documented  cost  savings  and  achievements  directly  attributable  to  the  project,  it 
would  be  a  mistake  to  classify  SHARE  as  a  failure.  The  ongoing  lessons  that  are  being 
learned  by  PEO-IWS  personnel  will  pay  dividends  in  the  future  when  the  best  strategies 
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and  solutions  are  figured  out.  These  solutions  might  come  after  a  period  of  trial  and  error 
but  it  is  a  learning  period  PEO-IWS  and  the  DAS  as  a  whole  has  to  pass  through  before 
we  can  build  solid  foundations  for  an  infrastructure  that  supports  the  principles  of  OA 
and  open  business.  An  analogy  would  be  sending  green  troops  into  combat.  Only  after 
serving  on  the  battlefield  and  learning  from  many  lethal  and  tragic  mistakes  will  a  combat 
organization  emerge  as  a  veteran  one.  Fortunately,  the  implementation  of  SHARE  is  not 
as  dramatic  and  the  time  spent  on  developing  the  best  implementation  strategy  is  a  small 
price  to  pay  to  build  the  expertise  necessary. 

D.  SUCCESSFUL  STRATEGIES  AND  RISK 

Certain  strategies  help  in  the  successful  outcomes  of  the  A-RCI  and  DE/TOSIP 
programs.  Some  of  the  strategies  are  the  same  as  the  best  practices  used  in  applying  SOA 
in  the  business  world  and  other  strategies  may  be  unorthodox.  This  section  will  explore 
the  strategies  used  by  the  A-RCI  and  DE/TOSIP  programs,  and  list  the  risks  taken  to 
achieve  program  success.  None  of  the  risks  taken  had  any  lasting  impact  of  these  two 
successful  programs  but  they  bear  attention  when  implementing  other  programs  that  may 
operate  under  different  conditions. 

1.  Best  Practices  in  Civilian  Industry 

A  researcher  working  on  a  companion  thesis  found  that  in  civilian  industry  the 
two  primary  benefits  of  adopting  SOA  were  decreased  risk  and  reusability  (Wolff,  2011). 
The  researcher  found  that  a  key  in  reducing  risk  was  the  flexibility  given  to  civilian 
management,  increasing  their  ability  to  react  to  unforeseen  events,  a  changing  market, 
and  evolving  technologies.  The  researcher  found  that  in  surveys  taken  by  civilian 
management  that  flexibility  was  -ahnost  always  at,  or  near  the  top  of  the  list  of  objectives 
when  implementing  SOA.” 

Reusability  was  key  in  many  successful  business  strategies  as  the  use  of  proven 

technologies  increased  the  availability  and  stability  factors  of  a  system.  Reusability 

combined  with  an  incremental  approach  in  the  implementation  of  SOA  proved  to  be  a 

winning  combination.  A  strategy  of  -attacking  the  low  hanging  fruit”  first  and  realizing 

the  easy  savings  in  costs  and  efficiencies  was  most  effective  (Wolff,  2011).  Most 
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companies  did  not  launch  overly  ambitious  projects  but  focused  on  the  specific  areas  that 
were  most  problematic.  This  approach  also  provided  an  added  benefit  in  allowing  an 
organization  to  leam  from  early  mistakes  in  the  first  development  cycles. 

2.  A-RCI  Best  Practices  and  Risk 

The  heavy  use  of  a  COTS  strategy  is  similar  to  the  civilian  industry  best  practices 
of  using  available,  proven,  state  of  the  art  technology  and  was  one  of  the  keys  to  the 
rapidly  improvement  in  sonar  technology.  The  risks  involved  for  a  DAS  program  to  use 
this  strategy  are  bureaucratic  and  security  risks.  Bureaucratic  in  that  the  COTS 
components  would  need  to  be  approved  in  both  the  JCIDS  process  and  by  the  end  to  end 
testing  requirements.  In  A-RCI’s  case,  the  sense  of  crisis  over  the  loss  of  technological 
superiority  in  a  vital  warfare  area  allowed  the  PMs  to  mitigate  this  risk  by  providing  top 
cover  to  their  middle  management  and  allowed  them  to  proceed  faster  than  the  testing 
and  JCIDS  cycle  would  allow  a  regular  program,  indicating  that  these  processes  may 
introduce  impediments  into  the  acquisition  community. 

The  open  capabilities  based  model  allow  A-RCI  to  leverage  the  rapid  gains  in 
computer  processing  power  that  civilian  industry  was  putting  to  good  use  at  the  time. 
This  was  contrary  to  the  requirement  based  model  that  lies  at  the  heart  of  DoD’s 
transfonnation  to  a  joint  operating  environment.  In  the  rush  to  develop  improved  sonar 
technologies,  the  program  introduced  future  integration  risks.  In  A-RCI’ s  case,  the 
system  they  were  developing  would  be  the  one  that  follow  on  systems  deployed  to  the 
surface  forces  and  aviation  assets  would  have  to  integrate.  The  integration  risks  may 
prove  too  much  for  the  rapid  development  of  a  system  already  tied  in  with  numerous 
legacy  weapons  systems. 

A  COTS  strategy  can  also  introduce  security  risks  when  developing  a  military 
system,  especially  a  system  that  uses  sensitive  technology  or  is  designated  Secret  or 
above.  The  security  risk  can  be  in  procuring  software  code  with  an  unknown  Trojan 
Horse  hidden  in  the  code  to  reliance  on  hardware  that  many  not  be  manufactured  to 
military  specifications. 
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A-RCI’s  spiral  development  strategy  opened  the  door  to  operational  risks,  and 
potential  costs  and  schedule  risks.  When  following  an  incremental  development  strategy 
technologies  will  be  released  when  they  are  -good  enough”  or  are  left  behind  to  proceed 
in  the  next  cycle.  The  cost  and  schedule  risks  arise  when  the  system  is  never  -good 
enough”  for  the  end  users  or  the  development  cycles  stagnate  into  repetitive  cycles  with 
little  gain.  The  operational  risk  was  born  by  the  submarine  fleet  in  the  initial  deployment 
of  the  sonar  system  but  was  never  viewed  as  presenting  too  big  of  a  challenge.  Prior  to 
implementing  the  program  A-RCI’s  PMs  had  a  fair  idea  of  the  current  state  of  the  civilian 
technology  available  and  the  cost  and  schedule  risk  was  deemed  acceptable.  The 
probability  was  that  they  would  be  able  to  achieve  rapid  improvement  over  the  legacy 
system. 


3.  DE/TOSIP  Best  Practices  and  Risk 

DE/TOSIP’s  main  strategy  was  to  avoid  the  requirements  of  the  DAS  altogether 
by  splitting  the  program  into  three  smaller  programs  and  avoiding  the  budget  threshold 
that  would  require  a  PM.  The  main  risks  to  such  a  strategy  are  scope  creep  and  budget 
risk.  As  the  program  was  being  developed,  DE/TOSIP’s  IPT  had  to  constantly  contend 
with  increase  user  requirements.  As  the  user  requirements  built  up  the  scope  of  the 
program  eventually  surpassed  the  budget  threshold  and  the  program  is  now  operating 
under  the  guidance  of  a  PM.  Additionally  the  program  would  be  at  risk  of  low  scalability 
as  a  program  that  was  split  up  and  developed  by  smaller  sub  programs  would  not  be  able 
to  rapidly  meet  unforeseen  expansion  requirements.  Ironically,  the  very  success  of  the 
DE/TOSIP  caused  it  to  grow  to  the  point  that  the  flexibility  and  initiative  program 
management  enjoyed  with  the  smaller  program  was  soon  lost. 

DE/TOSIP  used  an  extreme  COTS  strategy  in  the  decision  to  use  Oracle  products 
for  their  solution.  This  turned  out  to  work  well  for  program  success  but  does  introduce 
the  risk  on  relying  on  a  single  vendor.  In  DE/TOSIP’s  case,  it  would  not  have  been  cost 
effective  and  timely  to  try  and  introduce  a  collaboration  and  competition  based  strategy 
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when  the  solution  was  readily  at  hand,  but  the  single  vendor  solution  might  introduce  too 
much  friction  with  various  mandates,  regulations,  and  laws  in  the  DAS  for  larger 
programs 

DE/TOSIP’s  rapid  development  cycle  saved  costs  and  allowed  for  the  rapid 
delivery  of  the  solution  to  the  end  users  but  at  the  expense  of  performance  risk.  After  the 
system  was  deployed  and  gained  widespread  use,  requirement  complexity  increased  and 
eventually  slowed  the  development  cycle  down  from  one  year  increments  to  over  three 
years.  Due  to  the  need  to  keep  the  program  under  a  budget  threshold,  the  DE/TOSIP 
development  team  was  relatively  small.  The  small  team  was  constantly  at  risk  of  being 
overwhelmed.  It  was  the  COTS  strategy  and  heavy  reuse  of  components  that  allowed  the 
team  to  keep  up  with  the  three  month  development  cycles  (Maravola,  2009). 

Table  4  lists  the  successful  strategies  and  best  practices  used  by  the  A-RCI  and 
DE/TOSIP  programs  alongside  the  risks  that  can  be  introduced  when  pursing  them. 
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Best  Practices  and  Successful  Strategies 

Introduced  Risks 

A-RCI 

COTS 

Security  Risk 

Program  Risk  (bureaucratic  friction) 

Incremental  Strategy 

Operational  Risk  (initial  deployed  system 

does  not  meet  user  requirements) 

Cost  Risk 

Schedule  Risk 

MOSA 

Integration  Risk 

Open  Capabilities  Based  Model 

Integration  Risk 

Program  Risk  (bureaucratic  friction) 

DE/TOSIP 

Avoid  DAS  budget  threshold 

Risks  to  future  scalability 

Budget  risk 

COTS 

Program  Risk  (reliance  on  single  vendor) 

Program  Risk  (lack  of  collaboration  and 

cooperation  in  larger  programs) 

Rapid  Development  Increments 

Perfonnance  Risk 

Schedule  Risk 

Table  4.  Best  Practices  and  Risk 
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V.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  CONCLUSION 

This  thesis  explored  various  risks  and  uncertainties  in  DAS  to  determine  if  using 
OA  and  SO  A  principles  in  a  project  or  program  increases  or  decreases  risk  and 
uncertainties,  or  presents  the  OA  practitioner  with  unique  risks.  Additionally,  this  thesis 
attempted  to  detennine  if  OA  and  SOA  has  delivered  projected  cost  savings  and 
increased  efficiencies  to  DAS. 

1.  O  A/SO  A  and  Risk 

During  the  interview  process,  it  was  discovered  that  the  nature  of  DAS  does  not 
present  PMs  with  different  types  of  risk  at  different  stages  of  their  careers.  Beyond  cost, 
schedule,  and  performance  risks,  there  are  no  strict  definitions  of  different  types  of  risks 
or  a  specific  definition  of  uncertainty. 

Despite  the  lack  of  specific  definitions  for  different  types  of  risk,  there  are  risk 
areas  of  constant  concern  to  PMs.  Although  these  risks  may  be  classified  in  slightly 
different  ways  and  have  different  mitigation  strategies,  budget  and  program  risk  are  a 
constant  presence  in  the  DAS.  The  risks  associated  with  misunderstandings  between 
different  functional  areas,  the  -talking  by  each  other”  and  linguistic  discontinuity,  are  not 
unique  to  the  DAS  but  are  increased  by  a  lack  of  training  emphasizing  other  functional 
areas  of  the  DAS  along  with  contradictory  structural  goals  between  functional  areas  of  a 
program  such  as  between  a  PM  and  a  contracting  officer. 

The  findings  concerning  the  constant  nature  of  risk  throughout  a  PM’s  career  and 
widespread  agreements  amongst  PMs  of  the  nature  of  risk  in  the  DAS  helped  lead  the 
researcher  to  the  conclusion  that  the  DAS  is  highly  structured,  consisting  of  well  defined 
requirements  and  milestones.  There  is  little  incentive  for  a  PM’s  initiative  in  running 
projects  or  programs  and  little  room  for  flexibility  in  the  personnel  composition  of  their 
work  force.  The  DAS’s  bureaucratic  nature  is  a  constant  and  not  viewed  as  a  risk  but  it 
introduces  or  amplifies  risk  as  many  PMs  develop  a  fatalistic  attitude  towards  risk, 
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(i.e.,  -we  have  to  play  with  the  hand  we  are  dealt  with”)  and  delay  ramifications  for 
incorrect  or  even  illegal  decisions  falls  on  the  program  long  after  the  original  participants 
have  transferred. 

Even  though  OA  has  delivered  cost  savings  and  allowed  faster  system 
development  in  certain  cases  it  has  also  increased  complexity  and  risk  for  programs  in 
meeting  the  DISA  network  accreditation  processes. 

2.  OA/SOA  and  Cost  Savings  in  the  DAS 

Changes  in  the  DAS  for  widespread  use  of  OA  and  SOA  principles  have  come  at 
an  upfront  cost  in  the  establishment  of  a  basic  supporting  infrastructure.  For  example,  the 
SHARE  repository  shows  how  unforeseen  issues  may  arise  when  trying  to  facilitate  OA 
principles  such  as  collaboration  and  software  reuse.  Future  programs  would  benefit  from 
the  lessons  learned  with  A-RCI’s  faced  paced,  spiral  development  cycles  and  the  JCIDS 
process.  In  order  to  implement  a  suggestion  that  JCIDS  address  A-RCI  like  programs, 
more  investments  must  be  made  in  reviewing  all  relevant  instructions  and  regulations  and 
make  necessary  updates,  additions,  or  changes,  without  an  adverse  effect  on  more 
traditional  programs. 

The  use  of  OA  principles  in  A-RCI  resulted  in  significant  cost,  schedule,  and 
even,  after  an  initial  choice  to  field  the  first  iteration  of  the  system  that  was  not  up  to  the 
Fleet’s  requirements,  performance  benefits.  A-RCI  proves  that  cost  and  schedule 
benefits  to  a  program  are  possible  with  OA,  especially  compared  with  a  theoretical 
program  developed  in  a  traditional  proprietary,  stove  piped  system. 

USMEPCOM’s  DE/TOSIP  program  shows  that  using  OA  strategies  and 
principles  can  help  mitigate  serious  integration  issues  as  the  DoD  slowly  moves  to 
towards  the  seamless  transmission  of  data  throughout  the  DoD  and  other  Federal 
agencies.  The  use  of  COTS  software  and  reliance  on  software  reuse  resulted  in  both 
initial  cost  savings  and  follow  on  savings  as  work  proceeded  on  new  features  to  the 
USMEPCOM  network. 
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B.  RECOMMENDATIONS  FOR  THE  DOD 

Below  are  several  recommendations  based  on  thesis  conclusions. 

1.  Provide  Adequate  Resources 

The  first  recommendation  is  to  continue  work  on  building  the  DoD  infrastructure 
and  ensure  any  new  initiatives  are  sufficiently  resourced.  The  SHARE  repository  gives  a 
warning  on  concerning  the  lack  of  resources,  in  SHARE’S  case  the  lack  of  personnel  and 
time  to  ensure  that  contractors  met  all  administrative  requirements  concerning  intellectual 
property  rights.  For  the  near  future,  the  DoD  and  DAS  will  face  budget  constraints  but 
must  avoid  the  temptation  to  implement  new  aspects  of  a  supporting  infrastructure 
without  assigning  the  proper  resources.  What  would  be  worse  than  a  lack  of  supporting 
infrastructure  for  OA  is  new  policies  and  procedures  that  eventually  devolve  into  a 
-e-heck  in  the  box”  or  paperwork  drill  due  to  the  lack  of  proper  enforcement  or  follow 
through. 

2.  Greater  PM  Initiative 

The  DAS  should  experiment  with  giving  PMs  more  initiative  in  running 
programs.  DAS  has  evolved  into  a  system  concentrating  on  performance  issues,  even  at 
the  expense  of  costs  and  schedule.  The  delivery  of  world  class  systems  to  our  operating 
forces  should  always  remain  a  priority  but  the  DoD  should  look  into  allowing  PMs  more 
flexibility  in  running  their  programs.  A-RCI  showed  how  taking  initial  perfonnance 
risks,  security  risks  using  a  COTS  strategy,  and  potential  cost  and  schedule  risks  using  a 
spiral  development  strategy  can  lead  to  program  success.  Senior  leadership  had  to 
provide  top  level  coverage  in  order  for  A-RCI  to  proceed,  even  when  not  in  sync  with 
JCIDS  requirements. 

3.  Continued  Accountability 

This  recommendation  addresses  the  delay  ramifications  for  bad  decisions  in  the 
DAS.  Many  PMs  inherit  programs  that  bought  initial  success  at  the  expense  of  future 
risks  and  stability.  If  the  program  is  of  such  a  length  that  a  PM  has  transferred  after 
improprieties  or  poor  decisions  are  uncovered,  then  as  long  as  the  PM  is  still  in 
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government  service,  the  PM  should  remain  accountable  for  his  decisions  in  older 
programs.  This  accountability  would  also  temper  excessive  risk  taking  if  PMs  were 
given  more  flexibility  in  running  their  programs. 

4.  Greater  Flexibility 

A  PM  allowed  flexibility  in  running  his  or  her  program  could  reap  the  benefits  of 
Joy’s  Law,  stating,  -no  matter  who  you  are,  most  of  the  smartest  people  work  for 
someone  else.”  Attributed  to  Sun  Microsystems’s  founder  Bill  Joy,  this  quote  not  only 
applies  to  the  tech  industry,  but  to  all  organizations  to  include  the  DAS  (Lakhani  & 
Panetta,  2007).  This  law  does  not  reflect  poorly  on  the  acquisition  professionals  working 
in  the  DAS  but  reflects  the  fact  that  there  is  a  lot  of  talent  outside  of  DAS  that  could  be 
tapped  into  with  a  more  flexible  approach  to  systems  development.  Open  source 
software  projects  such  as  of  Linux  offer  many  lessons  and  opportunities  for  PMs  willing 
to  risk  costs  and  effort  in  exchange  for  access  to  knowledge  and  techniques  outside  of  the 
DAS.  Innocentive.com  is  a  means  by  which  PMs  in  the  traditionally  closed  civilian 
pharmaceutical,  biotechnology,  consumer  goods,  and  high  technology  industries  help 
solve  specific  technical  problems  (Lakhani  &  Panetta,  2007).  PMs  can  post  the  problem 
and  the  cash  prize  they  will  award  the  person  or  company  that  offers  the  best  solution. 
Another  example  the  DAS  could  follow  is  TopCoder,  founded  in  2001,  that  consists  of  a 
community  of  programmers  that  compete,  as  well  as  collaborate,  in  creating  solutions  to 
problems  that  TopCoder’s  clients  had  identified.  The  main  means  that  TopCoder  uses  to 
solve  client’s  problems  is  to  set  up  contests  in  which  community  members  compete  for 
money  and  skill  ratings  (Lakhani,  Garvin,  &  Lonstein,  2010). 

DAS  PMs  running  programs  should  be  allowed  to  post  problems,  not  at  the  risk  of 
national  security  considerations,  or  the  DAS  should  consider  taking  advantage  of 
collaboration  with  the  vast  knowledge  base  in  the  world  outside  of  the  DAS  and  continue 
to  build  a  supporting  infrastructure  for  OA  and  SOA  by  launching  the  DAS’s  own 
version  ofInnovative.com  or  TopCoder. 
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5. 


New  Metrics 


Due  to  the  innovativeness  of  implementing  an  open  business  model  in  DoD 
acquisitions,  program  Managers  must  be  given  an  entirely  new  way  of  determining  new 
metrics,  perhaps  similar  to  what  SMEs  use  in  measuring  its  use  of  intellectual  capital. 
The  finance  industry  has  struggled  to  obtain  a  consistent,  accurate  method  for  valuing 
companies  based  on  their  intellectual  capital,  but  has  failed  to  produce  any  of  great 
promise.  Some  academics  advocate  the  use  of  Patent  Citation  Counts  but  others  describe 
the  method  as  inadequate  (King  &  Zeithaml,  2003;  Housel  &  Nelson,  2005).  Various 
methods  based  on  cost  reduction  and  avoidance  have  also  been  proposed  such  as  Book 
Value,  Discounted  Cash  Flows,  and  Market  Comparables  (Housel  &  Lorentz, 
Understanding  the  risk  of  intellectual  capital:  the  potential  IC  to  real  IC  to  conversion 
ratio,  2011).  However,  these  often  neglect  approaches  to  IC  other  than  traditional 
cost/benefit  analysis;  doing  so  neglects  the  source  of  innovative  IC  employees. 

Instead  of  these  inadequate  forms  of  valuation,  measuring  the  explicit  potential 
innovative  IC  to  realized  innovative  IC  conversion  ratio,  as  described  by  Housel  and 
Lorentz,  2011,  can  provide  dynamic  and  insightful  data  on  which  companies  are  better 
using  their  innovative  IC  as  well  as  indicate  whether  their  use  of  this  asset  is  growing  or 
remaining  stagnant.  Explicit  potential  intellectual  capital  is  the  knowledge  contained 
within  employees;  knowledge  that  can  be  exponentially  increased  through  education  and 
training  (Housel  &  Lorentz,  2011).  -Real  or  realized  IC  would  represent  an  individual’s 
ability  to  convert  this  available  potential  IC  into  observable  outputs  such  as  decisions, 
new  product  or  service  ideas,  suggested  productivity  improvements,  and  other  observable 
outputs”  (Housel  &  Lorentz,  Understanding  the  risk  of  intellectual  capital:  the  potential 
IC  to  real  IC  to  conversion  ratio,  2011). 

The  explicit  potential  to  realized  IC  conversion  ratio  is  based  on  the  physics 
concept  of  potential  and  kinetic  energy.  As  an  engine  nears  the  1:1  ratio  of  energy 
consumed  to  power  produced,  it  becomes  more  efficient  and  productive;  it  is  likely  that 
so  too  would  a  company  that  neared  a  full  utilization  of  its  most  important  asset, 
intellectual  capital.  The  explicit  potential  innovative  IC  to  realized  IC  conversion  ratio 

eliminates  the  problems  associated  with  traditional  valuation  processes  for  innovative 
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SMEs  because  it  deals  directly  with  their  intangible  assets.  Furthermore,  as  this  ratio  can 
be  reevaluated  over  time,  it  may  provide  the  DoD  with  a  reasonable  view  of  whether  their 
efficiency  is  increasing,  decreasing,  or  flat-lining. 

Implementing  this  conversion  ratio  in  the  DoD  acquisitions  environment  demands 
that  a  standard  unit  of  measurement  be  assigned  to  potential  and  realized  innovative  IC. 
One  way  of  determining  that  unit  is  through  Knowledge  Value  Added  theory.  -KVA 
analysis  produces  a  return-on-knowledge  (ROK)  ratio  to  estimate  the  value  added  by 
given  knowledge  assets  regardless  of  where  they  are  located”  (Housel  &  Bell,  2001). 
Unfortunately,  these  ratios  have  to  be  tailored  to  fit  the  industry  that  is  to  be  examined,  so 
there  are  no  current  standard  units  of  value  that  the  DoD  can  utilize  immediately,  and 
research  will  have  to  be  conducted  in  order  to  detennine  these  units  for  the  different 
sectors  fanned  by  the  acquisitions  process.  If  pursuing  the  open  architecture/business 
model  continues  to  remain  important  to  the  DoD,  this  research  will  provide  a  productive 
way  to  reduce  risk  and  cost,  without  stifling  the  innovation,  which  is  so  desired  by 
defense  leaders. 

6.  New  Study 

The  final  recommendation  is  to  conduct  a  study  on  which  DAS  areas  would 
benefit  from  OA  and  programs  hindered  by  implementing  OA  and  SOA  principles.  The 
conclusions  of  this  thesis  point  to  the  benefits  of  OA  in  addressing  cost  and  schedule 
issues  but  increased  risks  with  programs  related  to  security  and  highly  sensitive  military 
technologies. 

C.  RESEARCH  SHORTCOMINGS 

The  primary  shortcoming  of  this  thesis  was  the  relatively  limited  numbers  of 
subjects  interviewed,  in  conjunction  with  the  fact  that  only  half  of  the  interview  subjects 
had  more  than  passing  experience  with  OA.  There  was  also  limited  material  publically 
available  on  program  failures,  particularly  in  failed  programs  that  applying  OA 
principles. 
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D. 


RECOMMENDATIONS  FOR  FUTURE  STUDIES 


The  researcher  feels  that  the  following  recommendations  help  DAS  analyze  the 
full  impact  of  OA  and  SOA. 

1.  Identify  Best  Fit  for  OA 

Categorize  the  most  prevalent  programs  in  the  DAS.  For  example,  missile 
systems,  automotive  transportation,  aviation,  networking,  and  network  security  should  be 
categorized  in  an  attempt  to  determine  which  programs  and  projects  benefits  from  OA 
use  and  those  programs,  which  would  hinder  it. 

2.  Categorize  Best  Tactics  and  Strategies 

When  implementing  OA  (COTS,  software  reuse),  categorize  the  different 
strategies  used.  Then  categorize  the  different  tactics  used  to  implement  these  strategies. 
For  example,  under  software  reuse  the  tactic  would  be  to  write  all  code  with  reuse  in 
mind  or  only  concentrate  on  future  reuse  possibilities  for  the  software  interfaces. 

3.  Expand  Interview  Pool  and  Add  to  the  Case  Studies 

Expand  interview  subjects  from  different  PEOs  besides  PEO-IWS  for  insight  into 
how  widespread  OA  use  in  DAS.  Build  upon  initial  case  studies  in  this  thesis  to  cover 
different  DAS  areas  and  provide  either  support  or  contrary  evidence  to  the  conclusions  of 
this  thesis. 

4.  Conduct  Free-Flowing  Interviews 

During  the  interview  process,  it  was  discovered  that  all  interview  subjects 
responded  negatively  to  the  prospect  filling  in  a  questionnaire  or  participate  in  a  highly 
structured  interview.  This  proved  to  be  true  with  both  interview  subjects  working  in 
DAS  and  those  with  DAS  experience  but  currently  serving  in  the  academic  field.  All 
interview  subjects  preferred  a  free  flowing  conversation  and  a  few  refused  to  look  at  the 
initial  list  of  questions  presented  by  the  researcher.  Ultimately,  interview  subjects 
answered  many  of  the  prepared  questions  without  reading  from  a  script.  For  questions 
not  addressed  in  conversation,  it  was  easy  to  bring  them  up  before  the  interview  ended. 
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Another  benefit  from  a  free-flowing  conversation  was  that  many  points  were  bought  up 
that  would  not  have  come  to  the  researcher’s  attention  if  the  interview  was  kept  to  the 
researcher’s  original  questions  and  fonnat. 

5.  Expand  Number  and  Diversity  of  Participants 

Acquisition  of  research  data  was  inhibited  by  the  researcher’s  reliance  on  a 
limited  pool  of  interview  subjects.  Unless  interview  subjects  are  retired,  they  will  be 
busy  pursing  their  careers  and  have  limited  time  to  give  to  a  student  researcher.  A 
researcher  should  start  the  interview  process  as  early  as  possible  while  politely  and 
persistently  pursuing  the  research.  The  researcher  made  the  mistake  of  waiting  too  long 
for  information  and  should  have  used  the  valuable  waiting  time  seeking  out  new 
interview  subjects.  The  earlier  the  researcher  starts  the  interview  process,  the  more  time 
allowed  for  information  gathering  without  the  researcher  pestering  the  subjects  and  the 
greater  the  number  of  interviews  will  surely  lead  to  a  diversity  of  opinion  and  many 
insights. 


6.  Map  the  Application  and  Mitigation  of  Risk 

From  case  studies  of  OA  and  SOA  based  acquisition  programs,  map  areas  in 
which  PMs  choose  to  accept  risk  and  which  risks  they  decided  to  mitigate  against  to 
provide  a  visual  summary  useful  for  future  research.  This  visual  of  risk  types  associated 
with  certain  types  of  programs  and  various  tactics  applied  by  PMs  to  either  mitigate  or 
accept  risks  to  further  cost,  schedule  and  performance  goals  would  be  extremely  useful. 
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